-- Logs begin at Fri 2022-09-30 08:51:06 UTC, end at Fri 2022-09-30 10:03:04 UTC. -- Sep 30 09:21:01.792814 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Sep 30 09:21:01.800151 dv-cinder-master-3-smb devstack@keystone.service[81357]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 30 09:21:01 2022] *** Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: nodename: dv-cinder-master-3-smb Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: machine: x86_64 Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: clock source: unix Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: pcre jit disabled Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: detected number of CPU cores: 4 Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: current working directory: / Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: detected binary path: /usr/bin/uwsgi-core Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: your processes number limit is 47680 Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: your memory page size is 4096 bytes Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: detected max file descriptor number: 2048 Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: lock engine: pthread robust mutexes Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: thunder lock: enabled Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 30 09:21:01.805487 dv-cinder-master-3-smb devstack@keystone.service[81357]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 30 09:21:01.843850 dv-cinder-master-3-smb devstack@keystone.service[81357]: Python main interpreter initialized at 0x560962aff9c0 Sep 30 09:21:01.843850 dv-cinder-master-3-smb devstack@keystone.service[81357]: python threads support enabled Sep 30 09:21:01.843850 dv-cinder-master-3-smb devstack@keystone.service[81357]: your server socket listen backlog is limited to 100 connections Sep 30 09:21:01.843850 dv-cinder-master-3-smb devstack@keystone.service[81357]: your mercy for graceful operations on workers is 90 seconds Sep 30 09:21:01.844245 dv-cinder-master-3-smb devstack@keystone.service[81357]: mapped 403077 bytes (393 KB) for 2 cores Sep 30 09:21:01.844326 dv-cinder-master-3-smb devstack@keystone.service[81357]: *** Operational MODE: preforking *** Sep 30 09:21:01.844562 dv-cinder-master-3-smb devstack@keystone.service[81357]: *** uWSGI is running in multiple interpreter mode *** Sep 30 09:21:01.844562 dv-cinder-master-3-smb devstack@keystone.service[81357]: spawned uWSGI master process (pid: 81357) Sep 30 09:21:01.844775 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Sep 30 09:21:01.847075 dv-cinder-master-3-smb devstack@keystone.service[81357]: spawned uWSGI worker 1 (pid: 81371, cores: 1) Sep 30 09:21:01.847075 dv-cinder-master-3-smb devstack@keystone.service[81357]: spawned uWSGI worker 2 (pid: 81372, cores: 1) Sep 30 09:21:01.847075 dv-cinder-master-3-smb devstack@keystone.service[81357]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 30 09:21:06.147445 dv-cinder-master-3-smb devstack@keystone.service[81371]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:21:06.147445 dv-cinder-master-3-smb devstack@keystone.service[81371]: from cryptography.utils import int_from_bytes Sep 30 09:21:06.148617 dv-cinder-master-3-smb devstack@keystone.service[81372]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:21:06.148617 dv-cinder-master-3-smb devstack@keystone.service[81372]: from cryptography.utils import int_from_bytes Sep 30 09:21:06.148617 dv-cinder-master-3-smb devstack@keystone.service[81372]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:21:06.148617 dv-cinder-master-3-smb devstack@keystone.service[81372]: from cryptography.utils import int_from_bytes Sep 30 09:21:06.149560 dv-cinder-master-3-smb devstack@keystone.service[81371]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 30 09:21:06.149560 dv-cinder-master-3-smb devstack@keystone.service[81371]: from cryptography.utils import int_from_bytes Sep 30 09:21:06.199143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 30 09:21:06.199334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 30 09:21:06.199507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] command line args: [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 30 09:21:06.199774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 30 09:21:06.199950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ================================================================================ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 30 09:21:06.200205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] admin_token = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.200448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] config_dir = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.200634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.200816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] config_source = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.200892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 30 09:21:06.200993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] debug = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.201078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 30 09:21:06.201191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] command line args: [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 30 09:21:06.201275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.201495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 30 09:21:06.201596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.201719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ================================================================================ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 30 09:21:06.201834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.201936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] admin_token = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] config_dir = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] config_source = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_config_append = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] debug = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.202935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_dir = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.203950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_config_append = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_dir = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_file = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.204848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.205960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.206081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.206199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.206321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.206470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.206560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.206765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.206949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] publish_errors = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.207944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.208070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.208215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.208430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.208516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] standard_threads = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.208631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] publish_errors = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.208712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.208830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.208917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_journal = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_json = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_stderr = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_syslog = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.209963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] standard_threads = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.210108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.210226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.210305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.210489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.210610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.210741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.210860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.210939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_journal = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.211027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.211100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_json = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.211170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_stderr = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.211240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.211319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_syslog = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.211425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.211524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 30 09:21:06.211672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.211758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.211905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.211994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.212151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.212229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.212396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.212473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.212643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.212729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.212874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.212952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.213117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.213196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.213357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.213448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.213604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.213711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.213873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.213952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.214107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.214198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.214342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.214426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.214590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.214668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.214826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.214912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.215058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.215135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.215308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.215390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.215533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.215607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.215786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.215864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.216969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.217052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.217215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.217295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.217459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.217536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.217708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.217791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.217939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.218025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.218171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.218248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.218407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.218493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.218640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.218717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.218885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.219878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.220951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.221956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.222109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.222186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.222355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.222431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.222583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.222661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.222817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.222895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.223974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.224057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.224208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.224344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.224449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.224553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.224649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.224744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.224979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.225963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.226122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.226214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.226357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.226471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.226618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.226701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.226828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.226912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.227063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.227143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.227302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.227388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.227542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.227671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.227783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.227861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.228965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.229998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.230151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.230229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.230384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.230474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.230630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.230708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.230997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.231869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.232926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.233956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.234110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.234195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.234343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.234422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.234574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.234663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.234806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.234885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.external = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.235987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.236091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.236245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.236325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.236472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.password = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.236551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.236702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.token = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.236780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.236938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.237016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.external = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.237171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.237269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.237418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.237497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.237665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.237781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.237915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.password = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.token = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.238932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.239083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.239167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.239330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.239410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.239569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.239648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.239798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.239875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.240029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.240108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.240274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.240356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.240522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.240608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.240754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.240834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.241000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.241078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.241244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.241323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.242063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.242152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.242335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.242430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.242576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.242656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.242824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.242912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.243056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.243135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.243287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.243373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.243531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.243615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.243780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.243859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.244973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.245053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.245222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.245301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.245453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.245540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.245719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.245803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.245955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.246033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.246202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.246293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.246436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.246515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.246667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.246745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.246900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.247922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.248999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.249093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.249243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.249348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.249486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.249569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.249740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.249835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.249978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.250995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.251146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.251235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.251372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.251453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.251601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.251682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.251835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.251924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.252069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.252150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.252313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.252394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.252540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.252621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.252782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.252862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.253976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.254977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.255135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.255215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.255367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.255449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.255602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.255712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.255835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.255968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.256058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.256219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.256300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.256453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.256532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.256684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.256772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.256933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.257119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.257840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.257979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.258138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.258297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.258414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.258494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.258592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.258674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.258829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.258920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.259070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.259151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.259312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.259391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.259549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.259638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.259785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.259864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.260961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.261052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.261194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.261274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.261426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.261506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.261678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.261775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.261927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.262952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.263097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.263186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.263338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.263419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.263571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.263652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.263808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.263899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.264978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.265057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.265212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.265300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.265444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.265525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.265704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.265789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.265943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.266962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.267111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.267192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.267345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.267432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.267580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.267661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.267829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.267933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.268073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.268171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.268304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.268390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.268539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.268627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.268780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.268867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.269974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] role.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] role.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.270991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.271149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] role.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.271228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.271390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.271469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] role.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.271621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.271700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.271867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.271990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.272111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.272191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.272355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.272436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.272582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.272713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.272864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.272951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.273116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.273257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.273393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.273618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.273731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.273873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.273966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.274148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.274328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.274475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.274554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.274728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.274817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.275034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.275112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.275281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.275527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.275760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.275839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.276009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.276183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.276264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.276483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.276652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.276985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.277183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.277261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.277414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.277619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.277735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.277922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.278104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.278184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.278385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.278545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.278633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.278831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.278988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.279071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.279281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.279466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.279554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.279742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.279820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.279994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.280074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.280240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.280327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.280475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.280554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.280716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.280956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.281034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.281203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.281436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.281515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.281690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.281928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.282998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.283172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.283252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.283430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.283509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.283672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.283759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.283921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.283999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.284155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.284234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.284392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.284479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.284623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.284704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.284855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.284934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.285085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.285173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.285318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.285398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.285559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.285653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.285813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.285905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.286054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.286134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.286286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.286365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.286529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.286617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.286759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.286851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.287831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 30 09:21:06.287991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.288238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.288539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.288779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.289104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.289351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.289704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.289950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.290395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.290831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.291283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.291697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.292013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.292442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.292889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.293335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 30 09:21:06.293637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 30 09:21:06.317373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.317596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.317842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.320463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.320766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.321001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.326970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.329218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.355995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.356130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.356213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.356287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.409920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.410173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.459289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.459520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.459608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.459781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.617434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.617669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.617854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.617944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.618090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.618176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.618305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.618387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.618520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.618613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.618736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.618952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.642643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.642878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.642972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.643975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.644058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.644218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.655943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.656288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.669347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.669453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.669545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.669715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.669803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.669990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.670402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.670478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.670567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.670643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.670716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.670789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.670861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.670943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 30 09:21:06.687622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.687832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 30 09:21:06.688173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.688377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 30 09:21:06.688459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.688616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 30 09:21:06.688699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.688904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.688986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.689140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 30 09:21:06.689230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.689410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.689491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.689695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.689957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.690210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.699126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.699406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.699680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.699868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.699951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.700147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.700414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.700698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.706915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.707608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.709811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.710098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.710900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.711131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.711216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.711419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.711631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.711717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.711897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.711977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.712170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.712259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.712447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.712529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.712719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.712801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.712995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.713077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.713262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.713342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.713531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.713827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.735820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.736253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.737286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.737571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.737741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.738011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.742761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.743040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.743322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.743410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.743603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.743883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.749622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.749929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.750145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.750430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.755498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.755794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.755998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.756083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.756288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.756387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.756569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.756655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.756849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.756933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.757205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.757461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.757550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.757614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.757774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.758036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.773465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.773787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.773881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.774151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.777871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.778234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.781083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.781381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.781466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.781715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.785569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.785864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.788857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.789279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.792083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.792403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.792527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.792675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.792768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.792960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.793058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.793243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.793329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.793536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.793621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.793855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.809892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.810182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.810449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.810708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.816485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.816784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.817056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.817322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.817843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.819382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.821668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.821945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.824643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.824815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.825218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.825496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.825926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.826385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.827916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.828204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.830989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.831280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.831572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.831859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.832245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.836916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.837204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.837489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.837794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.838177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.838473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.838752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.843350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.843637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.843921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.844208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.844589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.844869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.845157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.854274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.857147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.860019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.860411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.861476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.862960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.863243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.863325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.863643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.866222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.868351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.869844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.870130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.870484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.870575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.871934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.874820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.875216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.875499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.875774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.876821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.878298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.881154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.881554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.881875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.882702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.884187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.884482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.884785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.890500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.890793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.891075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.891552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.891834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.892111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.892383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.892670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.892922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.893191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.893475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.893776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.894050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.894331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.894588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.894849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.895129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.898401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 30 09:21:06.898685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.899000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.899278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.899552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.899824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.900098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.900385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.900655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.900934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.901214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.901480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.902039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.902312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 30 09:21:06.931351 dv-cinder-master-3-smb devstack@keystone.service[81371]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560962aff9c0 pid: 81371 (default app) Sep 30 09:21:06.934650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9cd6-c9d6-438b-809d-182c0fba4c0d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:06.934935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9cd6-c9d6-438b-809d-182c0fba4c0d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:06.935206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c9cd6-c9d6-438b-809d-182c0fba4c0d None None] PATH_INFO: `/v3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:06.936627 dv-cinder-master-3-smb devstack@keystone.service[81372]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560962aff9c0 pid: 81372 (default app) Sep 30 09:21:06.939333 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Fri Sep 30 09:21:06 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:13.815293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85496e0-2b9a-4458-a49b-eb3f86ad1574 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:13.815798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85496e0-2b9a-4458-a49b-eb3f86ad1574 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:13.815798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85496e0-2b9a-4458-a49b-eb3f86ad1574 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:13.820673 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:13.825964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:13.826468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:13.826563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:13.896728 dv-cinder-master-3-smb devstack@keystone.service[81371]: INFO dbcounter [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] Registered counter for database keystone Sep 30 09:21:13.916723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG oslo_db.sqlalchemy.engines [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81371) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:21:13.922857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writer thread running {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:21:14.107062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG passlib.handlers.bcrypt [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81371) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 30 09:21:14.107375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG passlib.handlers.bcrypt [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81371) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 30 09:21:14.167565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:14.169783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:14.296324 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 30 09:21:14.846821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85496e0-2b9a-4458-a49b-eb3f86ad1574 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:14.919370 dv-cinder-master-3-smb devstack@keystone.service[81372]: INFO dbcounter [None req-f85496e0-2b9a-4458-a49b-eb3f86ad1574 None None] Registered counter for database keystone Sep 30 09:21:14.941400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG oslo_db.sqlalchemy.engines [None req-f85496e0-2b9a-4458-a49b-eb3f86ad1574 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81372) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 30 09:21:14.948096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writer thread running {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 30 09:21:15.070550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85496e0-2b9a-4458-a49b-eb3f86ad1574 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:15.074812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0798dcb-9d9c-4559-8194-fbaccbb4656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:15.075439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0798dcb-9d9c-4559-8194-fbaccbb4656e admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:15.075721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0798dcb-9d9c-4559-8194-fbaccbb4656e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:15.075993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0798dcb-9d9c-4559-8194-fbaccbb4656e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:15.084099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0798dcb-9d9c-4559-8194-fbaccbb4656e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:15.136648 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f0798dcb-9d9c-4559-8194-fbaccbb4656e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 30 09:21:15.136997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0798dcb-9d9c-4559-8194-fbaccbb4656e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:15.146354 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Fri Sep 30 09:21:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:21:15.153274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:15.170332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3ebec-e724-4a8f-9932-f15f01a4bab3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:15.174795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703a558-d075-487b-9bc3-f2e0d033bbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:15.175895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703a558-d075-487b-9bc3-f2e0d033bbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:15.176183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703a558-d075-487b-9bc3-f2e0d033bbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:15.176460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703a558-d075-487b-9bc3-f2e0d033bbf6 admin admin] PATH_INFO: `/v3/projects/9494127904354d908def6993c27761dd` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:15.178086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703a558-d075-487b-9bc3-f2e0d033bbf6 admin admin] RBAC: Authorizing `identity:get_project(project_id=9494127904354d908def6993c27761dd)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:15.230631 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f703a558-d075-487b-9bc3-f2e0d033bbf6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 30 09:21:15.230996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703a558-d075-487b-9bc3-f2e0d033bbf6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:15.235913 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Fri Sep 30 09:21:15 2022] GET /identity/v3/projects/9494127904354d908def6993c27761dd => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:17.431259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ab22-fd79-4d15-a475-acc1b6506bca None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:17.431783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ab22-fd79-4d15-a475-acc1b6506bca None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:17.431783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ab22-fd79-4d15-a475-acc1b6506bca None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:17.435984 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:17.446398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034eb036-cb6a-42a3-adf0-16127f341821 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:17.446839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034eb036-cb6a-42a3-adf0-16127f341821 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:17.446962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034eb036-cb6a-42a3-adf0-16127f341821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:17.469117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-034eb036-cb6a-42a3-adf0-16127f341821 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:17.470706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-034eb036-cb6a-42a3-adf0-16127f341821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:17.479926 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 30 09:21:17.792423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ab22-fd79-4d15-a475-acc1b6506bca None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:17.805371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ab22-fd79-4d15-a475-acc1b6506bca None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:17.809727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc7ec6-b70c-4496-9978-e0b8b1075209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:17.810328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7ec6-b70c-4496-9978-e0b8b1075209 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:17.810636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7ec6-b70c-4496-9978-e0b8b1075209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:17.810907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7ec6-b70c-4496-9978-e0b8b1075209 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:17.819222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc7ec6-b70c-4496-9978-e0b8b1075209 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:17.821175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc7ec6-b70c-4496-9978-e0b8b1075209 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:17.830895 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Fri Sep 30 09:21:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:21:17.837283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034eb036-cb6a-42a3-adf0-16127f341821 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:17.846693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034eb036-cb6a-42a3-adf0-16127f341821 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:17.851818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502e160-5dd2-4262-93e4-da0f3693848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:17.852439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e160-5dd2-4262-93e4-da0f3693848f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:17.852709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e160-5dd2-4262-93e4-da0f3693848f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:17.852981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e160-5dd2-4262-93e4-da0f3693848f admin admin] PATH_INFO: `/v3/users/22586ccbb1964437ab05abad3612640c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:17.854335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502e160-5dd2-4262-93e4-da0f3693848f admin admin] RBAC: Authorizing `identity:get_user(user_id=22586ccbb1964437ab05abad3612640c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:17.856181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502e160-5dd2-4262-93e4-da0f3693848f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:17.857748 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Fri Sep 30 09:21:17 2022] GET /identity/v3/users/22586ccbb1964437ab05abad3612640c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:20.110861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dc779-ab7b-445d-9215-70d898a2d692 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:20.111400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dc779-ab7b-445d-9215-70d898a2d692 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:20.111400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903dc779-ab7b-445d-9215-70d898a2d692 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:20.112731 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:20.118006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946f5c6-b87c-44e6-bf2c-8a8b871b1982 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:20.118414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946f5c6-b87c-44e6-bf2c-8a8b871b1982 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:20.118566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946f5c6-b87c-44e6-bf2c-8a8b871b1982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:20.136777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c946f5c6-b87c-44e6-bf2c-8a8b871b1982 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:20.137990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c946f5c6-b87c-44e6-bf2c-8a8b871b1982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:20.148109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9111c-773f-497b-81f6-76fa48a829c6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:20.148387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9111c-773f-497b-81f6-76fa48a829c6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:20.148657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9111c-773f-497b-81f6-76fa48a829c6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:20.150185 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:20.155195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:20.155476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:20.155750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:20.198025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG passlib.handlers.bcrypt [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81372) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 30 09:21:20.198414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG passlib.handlers.bcrypt [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81372) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 30 09:21:20.222295 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:20.249117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:20.250749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:20.348668 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:20.496173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946f5c6-b87c-44e6-bf2c-8a8b871b1982 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:20.520569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:20.599278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946f5c6-b87c-44e6-bf2c-8a8b871b1982 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:20.621321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:20.633547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb354ad-2ce9-4aa8-93f8-aa1fa75f5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:20.634210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb354ad-2ce9-4aa8-93f8-aa1fa75f5cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:20.634493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb354ad-2ce9-4aa8-93f8-aa1fa75f5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:20.634764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb354ad-2ce9-4aa8-93f8-aa1fa75f5cb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:20.645616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb354ad-2ce9-4aa8-93f8-aa1fa75f5cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4eb354ad-2ce9-4aa8-93f8-aa1fa75f5cb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:20.655164 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:20.657782 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:20.658654 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:21:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:20.662019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb354ad-2ce9-4aa8-93f8-aa1fa75f5cb5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:20.727225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd746bd-fef9-45b2-b3c7-325a437bbb04 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:20.729502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:20.744972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:20.763566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb354ad-2ce9-4aa8-93f8-aa1fa75f5cb5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:20.764501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b45a2-b731-4785-889a-e4d413394d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:20.765359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b45a2-b731-4785-889a-e4d413394d93 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:20.765656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b45a2-b731-4785-889a-e4d413394d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:20.765951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b45a2-b731-4785-889a-e4d413394d93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:20.797208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:20.797829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:20.798109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:20.798378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:20.799037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:20.799337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:20.800730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:20.812843 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:20.818624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:20.834115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b45a2-b731-4785-889a-e4d413394d93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:20.836212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b45a2-b731-4785-889a-e4d413394d93 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:20.919645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f67d81-3422-4f8b-a7e9-8d5ebc40d35c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:20.931020 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 09:21:20.936465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b45a2-b731-4785-889a-e4d413394d93 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:20.939095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:20.953460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bfbf0-45b4-41e6-98b8-e846ef8fd90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:20.954189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfbf0-45b4-41e6-98b8-e846ef8fd90c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:20.954383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfbf0-45b4-41e6-98b8-e846ef8fd90c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:20.954649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bfbf0-45b4-41e6-98b8-e846ef8fd90c None admin] PATH_INFO: `/v3/users/22586ccbb1964437ab05abad3612640c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:20.955766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bfbf0-45b4-41e6-98b8-e846ef8fd90c None admin] RBAC: Authorizing `identity:get_user(user_id=22586ccbb1964437ab05abad3612640c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:20.957210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bfbf0-45b4-41e6-98b8-e846ef8fd90c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:20.958845 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:20 2022] GET /identity/v3/users/22586ccbb1964437ab05abad3612640c => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:20.964260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bfbf0-45b4-41e6-98b8-e846ef8fd90c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:20.978774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:21.035281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b45a2-b731-4785-889a-e4d413394d93 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:21.061979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bfbf0-45b4-41e6-98b8-e846ef8fd90c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:21.068489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:21.069530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:21.069845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:21.070121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:21.088702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:21.090208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:21.095287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2405a-3abc-4689-85bb-9c9106b5d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:21.095914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2405a-3abc-4689-85bb-9c9106b5d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:21.096191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2405a-3abc-4689-85bb-9c9106b5d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:21.096465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2405a-3abc-4689-85bb-9c9106b5d6a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:21.097567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2405a-3abc-4689-85bb-9c9106b5d6a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:21.099023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2405a-3abc-4689-85bb-9c9106b5d6a3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:21.101218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:21.107079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2405a-3abc-4689-85bb-9c9106b5d6a3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:21:21.108731 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:21.109603 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:21:21.110434 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:21:21.110434 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:21:21.110434 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:21.110434 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:21:21.115304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:21.172770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:21.187950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:21.207193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2405a-3abc-4689-85bb-9c9106b5d6a3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:21.218825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df45008-6697-42d0-8393-fd2ab50a8606 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:21.240577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502fcd3-38c8-46db-b81e-67c525032ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:21.241722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502fcd3-38c8-46db-b81e-67c525032ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:21.242009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502fcd3-38c8-46db-b81e-67c525032ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:21.242289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502fcd3-38c8-46db-b81e-67c525032ac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:21.243010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502fcd3-38c8-46db-b81e-67c525032ac6 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.domain.id=default, user.id=22586ccbb1964437ab05abad3612640c) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:21.243320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502fcd3-38c8-46db-b81e-67c525032ac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:21.244790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502fcd3-38c8-46db-b81e-67c525032ac6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:21.251304 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Fri Sep 30 09:21:21 2022] GET /identity/v3/role_assignments?user.id=22586ccbb1964437ab05abad3612640c&scope.domain.id=default&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:21.251449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2f434-3b73-410c-a496-278897234437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:21.252087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2f434-3b73-410c-a496-278897234437 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:21.252423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2f434-3b73-410c-a496-278897234437 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:21.252697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2f434-3b73-410c-a496-278897234437 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:21.253338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2f434-3b73-410c-a496-278897234437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:21.253658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2f434-3b73-410c-a496-278897234437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:21.255064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2f434-3b73-410c-a496-278897234437 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:21.274948 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:23.581011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d59361-e062-4cca-a6ec-122fdb641894 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:23.581507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d59361-e062-4cca-a6ec-122fdb641894 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:23.581604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d59361-e062-4cca-a6ec-122fdb641894 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:23.583056 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:23.589101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98049e-297b-4273-8131-b85235acc8dd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:23.589581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98049e-297b-4273-8131-b85235acc8dd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:23.589677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98049e-297b-4273-8131-b85235acc8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:23.614633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-df98049e-297b-4273-8131-b85235acc8dd None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:23.615923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-df98049e-297b-4273-8131-b85235acc8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:23.701540 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:23.971813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25b12-e933-4048-a25a-14c12f805082 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:23.972155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25b12-e933-4048-a25a-14c12f805082 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:23.972328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25b12-e933-4048-a25a-14c12f805082 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:23.973619 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:23.987510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb78dc-a574-4efc-ab63-bbe877904b73 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:23.987808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb78dc-a574-4efc-ab63-bbe877904b73 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:23.988074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb78dc-a574-4efc-ab63-bbe877904b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.016510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a7cb78dc-a574-4efc-ab63-bbe877904b73 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:24.017734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a7cb78dc-a574-4efc-ab63-bbe877904b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:24.072854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:24.107117 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 12/25] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:24.188965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed25b12-e933-4048-a25a-14c12f805082 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:24.284004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76e5f-aa70-465f-80e8-83c5cc64faa8 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:24.284286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76e5f-aa70-465f-80e8-83c5cc64faa8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:24.284485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76e5f-aa70-465f-80e8-83c5cc64faa8 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.285965 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 13/26] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:24.291968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb09be-0454-40ff-9145-30fda0a5894d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:24.292256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb09be-0454-40ff-9145-30fda0a5894d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:24.292531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb09be-0454-40ff-9145-30fda0a5894d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.326662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d6fb09be-0454-40ff-9145-30fda0a5894d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:24.327862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d6fb09be-0454-40ff-9145-30fda0a5894d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:24.333428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed25b12-e933-4048-a25a-14c12f805082 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:24.388658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf902964-4bcd-4eb6-9101-bf2de70459d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:24.389305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf902964-4bcd-4eb6-9101-bf2de70459d8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:24.389591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf902964-4bcd-4eb6-9101-bf2de70459d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:24.389895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf902964-4bcd-4eb6-9101-bf2de70459d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.413684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf902964-4bcd-4eb6-9101-bf2de70459d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-cf902964-4bcd-4eb6-9101-bf2de70459d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:24.421911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:24.422936 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:24.423834 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 14/27] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:21:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:24.430514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf902964-4bcd-4eb6-9101-bf2de70459d8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:24.446010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:24.523106 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 14/28] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:24.525030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb09be-0454-40ff-9145-30fda0a5894d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:24.632434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf902964-4bcd-4eb6-9101-bf2de70459d8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:24.693435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb26249-a749-4c29-92ff-c37c9e965356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:24.694625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26249-a749-4c29-92ff-c37c9e965356 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:24.694963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26249-a749-4c29-92ff-c37c9e965356 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:24.695306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26249-a749-4c29-92ff-c37c9e965356 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.695975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb26249-a749-4c29-92ff-c37c9e965356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:24.696278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb26249-a749-4c29-92ff-c37c9e965356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:24.697701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb26249-a749-4c29-92ff-c37c9e965356 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:24.707767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb09be-0454-40ff-9145-30fda0a5894d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:24.715665 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 15/29] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:24.721353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c5c53-f9a1-445b-9cd4-1a25012e555f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:24.721890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c5c53-f9a1-445b-9cd4-1a25012e555f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:24.724545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c5c53-f9a1-445b-9cd4-1a25012e555f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.726181 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 16/30] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:24.727040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c5c53-f9a1-445b-9cd4-1a25012e555f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:24.727562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:24.778667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9830e34-a78d-4a0b-ac85-39204b8bd5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:24.779759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9830e34-a78d-4a0b-ac85-39204b8bd5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:24.781263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9830e34-a78d-4a0b-ac85-39204b8bd5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:24.781935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9830e34-a78d-4a0b-ac85-39204b8bd5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.799164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9830e34-a78d-4a0b-ac85-39204b8bd5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:24.802810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9830e34-a78d-4a0b-ac85-39204b8bd5b6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a9830e34-a78d-4a0b-ac85-39204b8bd5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:21:24.829551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:24.830462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:21:24.831239 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:21:24.831239 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:21:24.831239 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:24.832778 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 15/31] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:24.834135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3035b7-9030-4c95-b697-59961b661d0f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:24.834555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3035b7-9030-4c95-b697-59961b661d0f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:24.835546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3035b7-9030-4c95-b697-59961b661d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.854597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:24.863857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ee3035b7-9030-4c95-b697-59961b661d0f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:24.866896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ee3035b7-9030-4c95-b697-59961b661d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:24.893610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c5c53-f9a1-445b-9cd4-1a25012e555f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:24.929961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c7545-3f34-40f2-b2a6-767c0ad2823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:24.930602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c7545-3f34-40f2-b2a6-767c0ad2823e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:24.930885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c7545-3f34-40f2-b2a6-767c0ad2823e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:24.931160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c7545-3f34-40f2-b2a6-767c0ad2823e None admin] PATH_INFO: `/v3/users/22586ccbb1964437ab05abad3612640c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:24.932298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c7545-3f34-40f2-b2a6-767c0ad2823e None admin] RBAC: Authorizing `identity:get_user(user_id=22586ccbb1964437ab05abad3612640c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:24.933867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c7545-3f34-40f2-b2a6-767c0ad2823e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:24.935686 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 17/32] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:24 2022] GET /identity/v3/users/22586ccbb1964437ab05abad3612640c => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:24.936529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c7545-3f34-40f2-b2a6-767c0ad2823e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:24.944624 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 16/33] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:24.945942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3035b7-9030-4c95-b697-59961b661d0f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:25.040150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:25.054703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3035b7-9030-4c95-b697-59961b661d0f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:25.091692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8dc5-3486-48c5-b0cb-f86b269623dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:25.092770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8dc5-3486-48c5-b0cb-f86b269623dc None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.093258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8dc5-3486-48c5-b0cb-f86b269623dc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.093751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8dc5-3486-48c5-b0cb-f86b269623dc None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.094620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f8dc5-3486-48c5-b0cb-f86b269623dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:25.096380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f8dc5-3486-48c5-b0cb-f86b269623dc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:25.110789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c7545-3f34-40f2-b2a6-767c0ad2823e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:25.146205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:25.146718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.147005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.147367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.148026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:25.148330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:25.149739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:25.160507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:25.177658 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 18/34] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:25.177658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:25.292668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de449b1-66d1-4437-80e0-3b01e02173e7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:25.317403 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 17/35] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 30 09:21:24 2022] POST /identity/v3/roles => generated 219 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:25.318613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8dc5-3486-48c5-b0cb-f86b269623dc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:25.355748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2f0c7-ecf8-4c54-b18b-168660307f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:25.356501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2f0c7-ecf8-4c54-b18b-168660307f95 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.356702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2f0c7-ecf8-4c54-b18b-168660307f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.356978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2f0c7-ecf8-4c54-b18b-168660307f95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.358297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2f0c7-ecf8-4c54-b18b-168660307f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:25.359832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2f0c7-ecf8-4c54-b18b-168660307f95 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:25.361872 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 19/36] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:25.363095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f35fb5-bbf4-4f2d-a4da-cccb48121936 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.363381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f35fb5-bbf4-4f2d-a4da-cccb48121936 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.363658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f35fb5-bbf4-4f2d-a4da-cccb48121936 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.364917 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 20/37] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:25.365610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f35fb5-bbf4-4f2d-a4da-cccb48121936 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:25.443859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:25.448652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8dc5-3486-48c5-b0cb-f86b269623dc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:25.490009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cb6dc-400f-4e44-96cb-023a3d53a9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:25.491588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb6dc-400f-4e44-96cb-023a3d53a9a1 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.492110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb6dc-400f-4e44-96cb-023a3d53a9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.492566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb6dc-400f-4e44-96cb-023a3d53a9a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.493719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cb6dc-400f-4e44-96cb-023a3d53a9a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:25.496016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cb6dc-400f-4e44-96cb-023a3d53a9a1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:25.499164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f35fb5-bbf4-4f2d-a4da-cccb48121936 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:25.524484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:25.538432 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 18/38] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:21:25 2022] POST /identity/v3/projects => generated 301 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:25.540613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0e84-116c-48e4-834c-256a32454d7e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.541194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0e84-116c-48e4-834c-256a32454d7e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.541818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e0e84-116c-48e4-834c-256a32454d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.550424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be7757-92ee-46f5-b512-ce1eb3d3cfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:25.551241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be7757-92ee-46f5-b512-ce1eb3d3cfd3 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.551624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be7757-92ee-46f5-b512-ce1eb3d3cfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.551997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be7757-92ee-46f5-b512-ce1eb3d3cfd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.552813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be7757-92ee-46f5-b512-ce1eb3d3cfd3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:25.555023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be7757-92ee-46f5-b512-ce1eb3d3cfd3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:25.565222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-210e0e84-116c-48e4-834c-256a32454d7e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:25.567237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-210e0e84-116c-48e4-834c-256a32454d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:25.656230 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 19/39] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:25.657016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e0e84-116c-48e4-834c-256a32454d7e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:25.777109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e0e84-116c-48e4-834c-256a32454d7e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:25.810826 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 21/40] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 30 09:21:25 2022] POST /identity/v3/roles => generated 225 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:25.815387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:25.819060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16537842-7720-431d-bc19-354be214d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:25.819849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537842-7720-431d-bc19-354be214d417 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.820074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537842-7720-431d-bc19-354be214d417 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.820316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537842-7720-431d-bc19-354be214d417 None admin] PATH_INFO: `/v3/domains/default/users/22586ccbb1964437ab05abad3612640c/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.823243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16537842-7720-431d-bc19-354be214d417 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=22586ccbb1964437ab05abad3612640c, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:25.825168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16537842-7720-431d-bc19-354be214d417 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:25.842955 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 20/41] 10.222.0.15 () {58 vars in 1361 bytes} [Fri Sep 30 09:21:25 2022] PUT /identity/v3/domains/default/users/22586ccbb1964437ab05abad3612640c/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:21:25.925812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6e060-135f-4354-b3e4-c966d2e7212c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.925999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6e060-135f-4354-b3e4-c966d2e7212c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.926280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6e060-135f-4354-b3e4-c966d2e7212c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.927725 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 22/42] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:25.934365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332cd4d-ae6a-4ce4-a1f7-a785666f1082 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:25.934731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332cd4d-ae6a-4ce4-a1f7-a785666f1082 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:25.934917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332cd4d-ae6a-4ce4-a1f7-a785666f1082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:25.956363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6332cd4d-ae6a-4ce4-a1f7-a785666f1082 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:25.957688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6332cd4d-ae6a-4ce4-a1f7-a785666f1082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:25.981758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6e060-135f-4354-b3e4-c966d2e7212c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:26.062552 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 21/43] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:26.121257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6e060-135f-4354-b3e4-c966d2e7212c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:26.149544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:26.164168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36d353-4b8d-4564-806a-8d03fa8dc903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:26.164951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d353-4b8d-4564-806a-8d03fa8dc903 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:26.165437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d353-4b8d-4564-806a-8d03fa8dc903 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:26.165763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36d353-4b8d-4564-806a-8d03fa8dc903 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:26.166466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36d353-4b8d-4564-806a-8d03fa8dc903 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:26.168003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36d353-4b8d-4564-806a-8d03fa8dc903 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:26.183356 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 23/44] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 30 09:21:25 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:21:26.339752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332cd4d-ae6a-4ce4-a1f7-a785666f1082 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:26.452725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332cd4d-ae6a-4ce4-a1f7-a785666f1082 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:26.479772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:26.497592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88285cd6-1d90-4918-ac63-2f52f9347a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:26.498258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88285cd6-1d90-4918-ac63-2f52f9347a61 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:26.498545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88285cd6-1d90-4918-ac63-2f52f9347a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:26.498824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88285cd6-1d90-4918-ac63-2f52f9347a61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:26.499953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88285cd6-1d90-4918-ac63-2f52f9347a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:26.501414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88285cd6-1d90-4918-ac63-2f52f9347a61 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:26.503011 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 22/45] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:26.509228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36d353-4b8d-4564-806a-8d03fa8dc903 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:26.607788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36d353-4b8d-4564-806a-8d03fa8dc903 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:26.641064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e7579-4845-4acd-9b0e-27af41f3a8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:26.641694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e7579-4845-4acd-9b0e-27af41f3a8e2 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:26.641975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e7579-4845-4acd-9b0e-27af41f3a8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:26.642247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e7579-4845-4acd-9b0e-27af41f3a8e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:26.642893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e7579-4845-4acd-9b0e-27af41f3a8e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:26.644351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e7579-4845-4acd-9b0e-27af41f3a8e2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:26.673578 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 24/46] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:21:26 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:28.131857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013591f-b08e-46df-a342-45eee173a084 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:28.132403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013591f-b08e-46df-a342-45eee173a084 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:28.132656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013591f-b08e-46df-a342-45eee173a084 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:28.134040 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 23/47] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:28.139386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1132c-6a2e-4d29-b323-1d38b6586345 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:28.139783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1132c-6a2e-4d29-b323-1d38b6586345 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:28.139959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1132c-6a2e-4d29-b323-1d38b6586345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:28.159168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-74a1132c-6a2e-4d29-b323-1d38b6586345 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:28.160680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-74a1132c-6a2e-4d29-b323-1d38b6586345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:28.187767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:28.229350 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 25/48] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:28.497466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013591f-b08e-46df-a342-45eee173a084 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:28.597322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013591f-b08e-46df-a342-45eee173a084 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:28.631379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee689c-0c93-46ea-82c8-09469aa88f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:28.632027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee689c-0c93-46ea-82c8-09469aa88f4d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:28.632304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee689c-0c93-46ea-82c8-09469aa88f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:28.632581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee689c-0c93-46ea-82c8-09469aa88f4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:28.644093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee689c-0c93-46ea-82c8-09469aa88f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-58ee689c-0c93-46ea-82c8-09469aa88f4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:28.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:28.652376 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:28.653111 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 24/49] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:21:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:28.658591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1132c-6a2e-4d29-b323-1d38b6586345 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:28.760404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1132c-6a2e-4d29-b323-1d38b6586345 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:28.792605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2974d-1284-4c45-9e1d-2cbb5cb5ac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:28.793210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2974d-1284-4c45-9e1d-2cbb5cb5ac79 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:28.793489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2974d-1284-4c45-9e1d-2cbb5cb5ac79 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:28.793792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2974d-1284-4c45-9e1d-2cbb5cb5ac79 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:28.794453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2974d-1284-4c45-9e1d-2cbb5cb5ac79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:28.794754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2974d-1284-4c45-9e1d-2cbb5cb5ac79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:28.796143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2974d-1284-4c45-9e1d-2cbb5cb5ac79 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:28.804750 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 26/50] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:28.810382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee689c-0c93-46ea-82c8-09469aa88f4d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:28.812106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637d035-fa09-4399-8213-4642dabd5ff2 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:28.812402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637d035-fa09-4399-8213-4642dabd5ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:28.812684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637d035-fa09-4399-8213-4642dabd5ff2 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:28.814038 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 27/51] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:28.820177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be760-d3db-4d9a-9ec5-8bcf8ea29d15 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:28.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be760-d3db-4d9a-9ec5-8bcf8ea29d15 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:28.820729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290be760-d3db-4d9a-9ec5-8bcf8ea29d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:28.838488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-290be760-d3db-4d9a-9ec5-8bcf8ea29d15 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:28.839673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-290be760-d3db-4d9a-9ec5-8bcf8ea29d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:28.859797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:28.859931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:28.905690 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 28/52] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:28.910770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee689c-0c93-46ea-82c8-09469aa88f4d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:28.942972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f60487-7974-4d39-8c47-64c3256a2e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:28.943601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f60487-7974-4d39-8c47-64c3256a2e24 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:28.944486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f60487-7974-4d39-8c47-64c3256a2e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:28.944871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f60487-7974-4d39-8c47-64c3256a2e24 None admin] PATH_INFO: `/v3/users/22586ccbb1964437ab05abad3612640c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:28.946027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f60487-7974-4d39-8c47-64c3256a2e24 None admin] RBAC: Authorizing `identity:get_user(user_id=22586ccbb1964437ab05abad3612640c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:28.947478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f60487-7974-4d39-8c47-64c3256a2e24 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:28.949038 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 25/53] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:28 2022] GET /identity/v3/users/22586ccbb1964437ab05abad3612640c => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:28.954521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290be760-d3db-4d9a-9ec5-8bcf8ea29d15 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:29.055372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290be760-d3db-4d9a-9ec5-8bcf8ea29d15 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:29.091991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dc20d-d3df-4d2d-a68a-a03d72cb1232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:29.092625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dc20d-d3df-4d2d-a68a-a03d72cb1232 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:29.092907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dc20d-d3df-4d2d-a68a-a03d72cb1232 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:29.093180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dc20d-d3df-4d2d-a68a-a03d72cb1232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:29.094289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3dc20d-d3df-4d2d-a68a-a03d72cb1232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:29.095747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3dc20d-d3df-4d2d-a68a-a03d72cb1232 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:29.097298 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 29/54] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:29.103089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f60487-7974-4d39-8c47-64c3256a2e24 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:29.172105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dc20d-d3df-4d2d-a68a-a03d72cb1232 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:29.200374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:29.282894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dc20d-d3df-4d2d-a68a-a03d72cb1232 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:29.306423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f60487-7974-4d39-8c47-64c3256a2e24 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:29.318364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de11383-ef9d-4f1f-8f5d-107d988ffb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:29.320031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11383-ef9d-4f1f-8f5d-107d988ffb8d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:29.320325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11383-ef9d-4f1f-8f5d-107d988ffb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:29.320604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11383-ef9d-4f1f-8f5d-107d988ffb8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:29.321757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de11383-ef9d-4f1f-8f5d-107d988ffb8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:29.323279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de11383-ef9d-4f1f-8f5d-107d988ffb8d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:29.323432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:29.324916 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 30/55] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:29.331419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de11383-ef9d-4f1f-8f5d-107d988ffb8d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:29.342116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988b36d-212e-4d50-8b47-d8fe191a1870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:29.342939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988b36d-212e-4d50-8b47-d8fe191a1870 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:29.343220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988b36d-212e-4d50-8b47-d8fe191a1870 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:29.343498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988b36d-212e-4d50-8b47-d8fe191a1870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:29.344220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988b36d-212e-4d50-8b47-d8fe191a1870 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.domain.id=default, user.id=22586ccbb1964437ab05abad3612640c) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:29.344522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988b36d-212e-4d50-8b47-d8fe191a1870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:29.345991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988b36d-212e-4d50-8b47-d8fe191a1870 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:29.353959 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 26/56] 10.222.0.15 () {56 vars in 1322 bytes} [Fri Sep 30 09:21:29 2022] GET /identity/v3/role_assignments?user.id=22586ccbb1964437ab05abad3612640c&scope.domain.id=default&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 530 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:29.430035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:29.439861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de11383-ef9d-4f1f-8f5d-107d988ffb8d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:29.478097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364c628-1c4c-4773-9cda-1491423eb202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:29.478720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364c628-1c4c-4773-9cda-1491423eb202 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:29.478999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364c628-1c4c-4773-9cda-1491423eb202 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:29.479278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364c628-1c4c-4773-9cda-1491423eb202 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:29.479944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364c628-1c4c-4773-9cda-1491423eb202 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:29.481443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364c628-1c4c-4773-9cda-1491423eb202 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:29.509451 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 31/57] 10.222.0.15 () {60 vars in 1140 bytes} [Fri Sep 30 09:21:29 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:21:31.661362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3013-cf9b-4fef-b818-c706ff3eb9d0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:31.661884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3013-cf9b-4fef-b818-c706ff3eb9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:31.661884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc3013-cf9b-4fef-b818-c706ff3eb9d0 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:31.663392 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 27/58] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:31.668986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b846f61-44e7-4948-8a8f-055aff70a6c0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:31.669353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b846f61-44e7-4948-8a8f-055aff70a6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:31.669538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b846f61-44e7-4948-8a8f-055aff70a6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:31.687266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9b846f61-44e7-4948-8a8f-055aff70a6c0 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:31.688455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9b846f61-44e7-4948-8a8f-055aff70a6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:31.751687 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 32/59] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:32.025704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3013-cf9b-4fef-b818-c706ff3eb9d0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:32.124106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc3013-cf9b-4fef-b818-c706ff3eb9d0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:32.160296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388b1fa-8132-48d6-8eae-7347be9662cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:32.160928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b1fa-8132-48d6-8eae-7347be9662cf None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:32.161206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b1fa-8132-48d6-8eae-7347be9662cf None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:32.161476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388b1fa-8132-48d6-8eae-7347be9662cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:32.162749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388b1fa-8132-48d6-8eae-7347be9662cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:32.164288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388b1fa-8132-48d6-8eae-7347be9662cf None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:32.165871 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 28/60] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:32.175912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b846f61-44e7-4948-8a8f-055aff70a6c0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:32.247377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:32.272712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b846f61-44e7-4948-8a8f-055aff70a6c0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:32.305898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de877b-fc96-4eca-9780-da9be9178331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:32.306438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de877b-fc96-4eca-9780-da9be9178331 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:32.306716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de877b-fc96-4eca-9780-da9be9178331 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:32.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de877b-fc96-4eca-9780-da9be9178331 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:32.307633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de877b-fc96-4eca-9780-da9be9178331 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:32.309075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de877b-fc96-4eca-9780-da9be9178331 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:32.334533 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 33/61] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:21:32 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:35.817569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79939-ac0b-4dd4-8456-be26ec27f76a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:35.817569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79939-ac0b-4dd4-8456-be26ec27f76a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:35.817569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f79939-ac0b-4dd4-8456-be26ec27f76a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:35.819231 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 29/62] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:35.823379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17c0e1-97ff-45c4-afef-621b70ef1431 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:35.823774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17c0e1-97ff-45c4-afef-621b70ef1431 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:35.826515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17c0e1-97ff-45c4-afef-621b70ef1431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:35.847938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bf17c0e1-97ff-45c4-afef-621b70ef1431 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:35.849415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bf17c0e1-97ff-45c4-afef-621b70ef1431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:35.873012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f7d36-bf9a-409e-9b1e-06ea3f3f1e78 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:35.873986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f7d36-bf9a-409e-9b1e-06ea3f3f1e78 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:35.874489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f7d36-bf9a-409e-9b1e-06ea3f3f1e78 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:35.876448 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 30/63] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:35.890876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2381-1724-46a8-bf5e-a1f7539d261b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:35.891473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2381-1724-46a8-bf5e-a1f7539d261b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:35.891942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2381-1724-46a8-bf5e-a1f7539d261b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:35.916104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2acc2381-1724-46a8-bf5e-a1f7539d261b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:35.917687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2acc2381-1724-46a8-bf5e-a1f7539d261b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:35.976332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:35.984195 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 34/64] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:35.994086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcd23-adac-4f9b-9e17-389c2768e635 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:35.994403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcd23-adac-4f9b-9e17-389c2768e635 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:35.997471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdcd23-adac-4f9b-9e17-389c2768e635 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:35.998983 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 35/65] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:35 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:36.004609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b45dbc-65cd-427a-b704-60cae7a7698e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.004896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b45dbc-65cd-427a-b704-60cae7a7698e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.005175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b45dbc-65cd-427a-b704-60cae7a7698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.030778 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 31/66] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:36.039804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f9b45dbc-65cd-427a-b704-60cae7a7698e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:36.041029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f9b45dbc-65cd-427a-b704-60cae7a7698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:36.131775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:36.146556 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 36/67] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:36.195156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b2982-b9aa-4050-9101-ce16fc08c012 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.195939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b2982-b9aa-4050-9101-ce16fc08c012 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.196419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b2982-b9aa-4050-9101-ce16fc08c012 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.198009 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 32/68] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:36.203243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0913-ce28-478d-b465-099116c95cc3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.203535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0913-ce28-478d-b465-099116c95cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.203815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0913-ce28-478d-b465-099116c95cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.225131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-77de0913-ce28-478d-b465-099116c95cc3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:36.226351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-77de0913-ce28-478d-b465-099116c95cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:36.315259 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 37/69] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:36.411823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b2982-b9aa-4050-9101-ce16fc08c012 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:36.465847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de0913-ce28-478d-b465-099116c95cc3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:36.550016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b2982-b9aa-4050-9101-ce16fc08c012 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:36.582351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de0913-ce28-478d-b465-099116c95cc3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:36.586294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb1b21-7de3-44c7-b839-e2c9ce9cfc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:36.586932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb1b21-7de3-44c7-b839-e2c9ce9cfc98 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.587332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb1b21-7de3-44c7-b839-e2c9ce9cfc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.587609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb1b21-7de3-44c7-b839-e2c9ce9cfc98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.597570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb1b21-7de3-44c7-b839-e2c9ce9cfc98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2fbb1b21-7de3-44c7-b839-e2c9ce9cfc98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:36.602995 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:36.603839 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:36.604600 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 33/70] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:21:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:36.606234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb1b21-7de3-44c7-b839-e2c9ce9cfc98 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:36.606895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:36.621796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce4b85-8ff4-4418-bc27-28f6046e1dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:36.622436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4b85-8ff4-4418-bc27-28f6046e1dac None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.622714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4b85-8ff4-4418-bc27-28f6046e1dac None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.622982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4b85-8ff4-4418-bc27-28f6046e1dac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.634737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce4b85-8ff4-4418-bc27-28f6046e1dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-49ce4b85-8ff4-4418-bc27-28f6046e1dac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:36.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:36.640375 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:36.641152 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 38/71] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:21:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:36.642196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ef461-25af-4756-a7ed-d606200d1e2a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.642478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ef461-25af-4756-a7ed-d606200d1e2a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.642750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ef461-25af-4756-a7ed-d606200d1e2a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.644034 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 39/72] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:36.644757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ef461-25af-4756-a7ed-d606200d1e2a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:36.651420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:36.712947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb1b21-7de3-44c7-b839-e2c9ce9cfc98 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:36.746148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ea32a-5969-4392-b995-ad53fdfd3e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:36.746777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ea32a-5969-4392-b995-ad53fdfd3e6a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.747054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ea32a-5969-4392-b995-ad53fdfd3e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.747324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ea32a-5969-4392-b995-ad53fdfd3e6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.750150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ef461-25af-4756-a7ed-d606200d1e2a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:36.758589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ea32a-5969-4392-b995-ad53fdfd3e6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-411ea32a-5969-4392-b995-ad53fdfd3e6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:36.763425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:36.764253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:36.764992 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 34/73] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:36.765608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ea32a-5969-4392-b995-ad53fdfd3e6a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:36.789830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:36.790440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.790724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.791000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.791670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:36.791975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:36.793374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:36.801185 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 40/74] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:36.801997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:36.843075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:36.870552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:36.875899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ea32a-5969-4392-b995-ad53fdfd3e6a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:36.902336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561d0a0-c28c-4b1d-9ae8-f6df62074e80 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:36.911132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b1662-1b87-439e-b07a-7dc9322857b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:36.911773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1662-1b87-439e-b07a-7dc9322857b0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.912052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1662-1b87-439e-b07a-7dc9322857b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.912329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b1662-1b87-439e-b07a-7dc9322857b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.922480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b1662-1b87-439e-b07a-7dc9322857b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a98b1662-1b87-439e-b07a-7dc9322857b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:36.927291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:36.928657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:36.929432 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 35/75] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:36.930327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f349-13ed-4eac-94ee-185aa01251b6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.930613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f349-13ed-4eac-94ee-185aa01251b6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.930896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f349-13ed-4eac-94ee-185aa01251b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.936354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:36.936960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:36.937241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:36.937512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:36.938198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:36.938503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:36.939893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:36.948407 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 41/76] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:21:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:36.949194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:36.949648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fb97f349-13ed-4eac-94ee-185aa01251b6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:36.950909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fb97f349-13ed-4eac-94ee-185aa01251b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:37.019195 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 36/77] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:37.023110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97f349-13ed-4eac-94ee-185aa01251b6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.047791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88a186-02b4-44e6-aedb-71c040811fef None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.051122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.088018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.088623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.088914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.089188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.089887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:37.090193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.091591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.099616 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 42/78] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:37.101351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.116288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.130347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97f349-13ed-4eac-94ee-185aa01251b6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.166564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218df72-3c2a-4769-8690-af061a6103f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.167223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218df72-3c2a-4769-8690-af061a6103f8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.167504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218df72-3c2a-4769-8690-af061a6103f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.167779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218df72-3c2a-4769-8690-af061a6103f8 None admin] PATH_INFO: `/v3/users/22586ccbb1964437ab05abad3612640c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.169340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218df72-3c2a-4769-8690-af061a6103f8 None admin] RBAC: Authorizing `identity:get_user(user_id=22586ccbb1964437ab05abad3612640c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.170824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218df72-3c2a-4769-8690-af061a6103f8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.176921 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 37/79] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/users/22586ccbb1964437ab05abad3612640c => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:37.176921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218df72-3c2a-4769-8690-af061a6103f8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.213419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449be0b-f86a-4e8c-a822-7aaafd30b69d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.253480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.254128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.254414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.254690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.255363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:37.255678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.257651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.266404 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 43/80] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:37.267204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.288998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218df72-3c2a-4769-8690-af061a6103f8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.296698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.323090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaa8b3-43cd-485f-bd16-546d0c0ecba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.323724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaa8b3-43cd-485f-bd16-546d0c0ecba0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.324010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaa8b3-43cd-485f-bd16-546d0c0ecba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.324286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaa8b3-43cd-485f-bd16-546d0c0ecba0 None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.340322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aaa8b3-43cd-485f-bd16-546d0c0ecba0 None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.341822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aaa8b3-43cd-485f-bd16-546d0c0ecba0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.343435 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 38/81] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:37.344199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaa8b3-43cd-485f-bd16-546d0c0ecba0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.363071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.368957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449faa59-bd53-4e6e-a894-882fedca6e5f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.404968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2b9ef-c5a0-4a00-bc5e-632e69bec0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.406084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2b9ef-c5a0-4a00-bc5e-632e69bec0ca None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.406377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2b9ef-c5a0-4a00-bc5e-632e69bec0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.407174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2b9ef-c5a0-4a00-bc5e-632e69bec0ca None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.408027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2b9ef-c5a0-4a00-bc5e-632e69bec0ca None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.409501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2b9ef-c5a0-4a00-bc5e-632e69bec0ca None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.411219 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 44/82] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:37.411950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2b9ef-c5a0-4a00-bc5e-632e69bec0ca None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.451076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaa8b3-43cd-485f-bd16-546d0c0ecba0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.485477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135beb-34aa-49d5-9159-0efd7d9e8796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.486102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135beb-34aa-49d5-9159-0efd7d9e8796 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.486383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135beb-34aa-49d5-9159-0efd7d9e8796 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.486656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135beb-34aa-49d5-9159-0efd7d9e8796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.505589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.512605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2b9ef-c5a0-4a00-bc5e-632e69bec0ca None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.548628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c60e-18df-4fe7-a416-59ac2b4b940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.549192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c60e-18df-4fe7-a416-59ac2b4b940f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.549475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c60e-18df-4fe7-a416-59ac2b4b940f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.549783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c60e-18df-4fe7-a416-59ac2b4b940f None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.550912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9c60e-18df-4fe7-a416-59ac2b4b940f None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.552366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9c60e-18df-4fe7-a416-59ac2b4b940f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.554988 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 45/83] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:37.555766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c60e-18df-4fe7-a416-59ac2b4b940f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.557604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf135beb-34aa-49d5-9159-0efd7d9e8796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.559106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf135beb-34aa-49d5-9159-0efd7d9e8796 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.574659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.653453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c60e-18df-4fe7-a416-59ac2b4b940f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.662211 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 39/84] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:37.662211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135beb-34aa-49d5-9159-0efd7d9e8796 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.688788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459c3cc-eab6-40bc-8be1-734b2da7398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.689412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459c3cc-eab6-40bc-8be1-734b2da7398c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.689730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459c3cc-eab6-40bc-8be1-734b2da7398c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.690012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459c3cc-eab6-40bc-8be1-734b2da7398c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.691108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459c3cc-eab6-40bc-8be1-734b2da7398c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.692562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459c3cc-eab6-40bc-8be1-734b2da7398c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.694218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 46/85] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:37.695044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459c3cc-eab6-40bc-8be1-734b2da7398c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.726695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.759705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135beb-34aa-49d5-9159-0efd7d9e8796 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.773759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.794626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaba026-e630-4b34-ae7d-87bbf48f8332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.795239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaba026-e630-4b34-ae7d-87bbf48f8332 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.795523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaba026-e630-4b34-ae7d-87bbf48f8332 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.795736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459c3cc-eab6-40bc-8be1-734b2da7398c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:37.795839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaba026-e630-4b34-ae7d-87bbf48f8332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.831291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee24b10-785c-44ea-948c-13167223934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:37.831819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24b10-785c-44ea-948c-13167223934c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:37.832098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24b10-785c-44ea-948c-13167223934c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:37.832370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee24b10-785c-44ea-948c-13167223934c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:37.861738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaba026-e630-4b34-ae7d-87bbf48f8332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.863252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaba026-e630-4b34-ae7d-87bbf48f8332 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.899309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee24b10-785c-44ea-948c-13167223934c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:37.900815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee24b10-785c-44ea-948c-13167223934c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:37.930476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:37.961516 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 40/86] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:37.962386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaba026-e630-4b34-ae7d-87bbf48f8332 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:37.979498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:38.001351 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 47/87] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:38.002230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee24b10-785c-44ea-948c-13167223934c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.063637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaba026-e630-4b34-ae7d-87bbf48f8332 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.097528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1d73b-6b4d-45b6-8e5b-961b3e73c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.098095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1d73b-6b4d-45b6-8e5b-961b3e73c610 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.098376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1d73b-6b4d-45b6-8e5b-961b3e73c610 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.098648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1d73b-6b4d-45b6-8e5b-961b3e73c610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.102093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee24b10-785c-44ea-948c-13167223934c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.131813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:38.137387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dc0e8-db2b-4acb-b22f-5305584fb078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.138041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc0e8-db2b-4acb-b22f-5305584fb078 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.138324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc0e8-db2b-4acb-b22f-5305584fb078 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.138608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc0e8-db2b-4acb-b22f-5305584fb078 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.139713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dc0e8-db2b-4acb-b22f-5305584fb078 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.141170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dc0e8-db2b-4acb-b22f-5305584fb078 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.142822 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 48/88] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:38 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:38.143561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dc0e8-db2b-4acb-b22f-5305584fb078 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.167701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1d73b-6b4d-45b6-8e5b-961b3e73c610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.169183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1d73b-6b4d-45b6-8e5b-961b3e73c610 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.184309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:38.244337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dc0e8-db2b-4acb-b22f-5305584fb078 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.271023 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 41/89] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:38.272212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1d73b-6b4d-45b6-8e5b-961b3e73c610 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.281549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c2f4e-a20f-433a-923a-ce1b7f4edf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.282207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c2f4e-a20f-433a-923a-ce1b7f4edf15 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.282511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c2f4e-a20f-433a-923a-ce1b7f4edf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.282794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c2f4e-a20f-433a-923a-ce1b7f4edf15 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.283450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c2f4e-a20f-433a-923a-ce1b7f4edf15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.284899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c2f4e-a20f-433a-923a-ce1b7f4edf15 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.317662 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 49/90] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:21:38 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:38.317662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c2f4e-a20f-433a-923a-ce1b7f4edf15 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.374250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:38.379262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1d73b-6b4d-45b6-8e5b-961b3e73c610 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.399639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:38.425273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d9a0-5656-435f-b36f-8ee33f623255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.425970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d9a0-5656-435f-b36f-8ee33f623255 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.426254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d9a0-5656-435f-b36f-8ee33f623255 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.426532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456d9a0-5656-435f-b36f-8ee33f623255 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.427719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456d9a0-5656-435f-b36f-8ee33f623255 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.429768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456d9a0-5656-435f-b36f-8ee33f623255 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.431482 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 42/91] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:38 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:38.432290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d9a0-5656-435f-b36f-8ee33f623255 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.444566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c2f4e-a20f-433a-923a-ce1b7f4edf15 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.483092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e6a20-9baa-4ccc-91dc-05353e096b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.483741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e6a20-9baa-4ccc-91dc-05353e096b75 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.484028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e6a20-9baa-4ccc-91dc-05353e096b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.484310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e6a20-9baa-4ccc-91dc-05353e096b75 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.485517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e6a20-9baa-4ccc-91dc-05353e096b75 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.487035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e6a20-9baa-4ccc-91dc-05353e096b75 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.488679 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 50/92] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:38 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:38.489452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e6a20-9baa-4ccc-91dc-05353e096b75 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.561241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456d9a0-5656-435f-b36f-8ee33f623255 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.605197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.605851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.606139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.606241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e6a20-9baa-4ccc-91dc-05353e096b75 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.606416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.607158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0, user.id=22586ccbb1964437ab05abad3612640c) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:38.607467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.609108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.618561 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 43/93] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:38 2022] GET /identity/v3/role_assignments?user.id=22586ccbb1964437ab05abad3612640c&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:38.619924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.636073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:38.657386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dab23-b4c9-48a2-ae7b-1e6ecd0a7c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.658120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dab23-b4c9-48a2-ae7b-1e6ecd0a7c25 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.658401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dab23-b4c9-48a2-ae7b-1e6ecd0a7c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.658672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160dab23-b4c9-48a2-ae7b-1e6ecd0a7c25 None admin] PATH_INFO: `/v3/projects/94e4dc5051034046b6daf8b6972d78f4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.659803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160dab23-b4c9-48a2-ae7b-1e6ecd0a7c25 None admin] RBAC: Authorizing `identity:get_project(project_id=94e4dc5051034046b6daf8b6972d78f4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.661282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160dab23-b4c9-48a2-ae7b-1e6ecd0a7c25 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.668397 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 51/94] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:38 2022] GET /identity/v3/projects/94e4dc5051034046b6daf8b6972d78f4 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:38.668397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dab23-b4c9-48a2-ae7b-1e6ecd0a7c25 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.686035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:38.758119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2a0fc-079f-40eb-882e-2ae11b45c046 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.783253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160dab23-b4c9-48a2-ae7b-1e6ecd0a7c25 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.805084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.805749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.806039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.806319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.807074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] RBAC: Adding query filter params (role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0, user.id=c28c6caabed44035add7bfb1c3325511) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:38.807390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.809808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.816733 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 44/95] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:38 2022] GET /identity/v3/role_assignments?user.id=c28c6caabed44035add7bfb1c3325511&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:38.817599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:38.825901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0afeab-051f-45e6-8d38-207bfafddbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:38.826551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0afeab-051f-45e6-8d38-207bfafddbbc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:38.826834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0afeab-051f-45e6-8d38-207bfafddbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:38.827150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0afeab-051f-45e6-8d38-207bfafddbbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:38.827900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0afeab-051f-45e6-8d38-207bfafddbbc None admin] RBAC: Adding query filter params (role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0, user.id=c28c6caabed44035add7bfb1c3325511) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:38.828210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0afeab-051f-45e6-8d38-207bfafddbbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:38.829748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0afeab-051f-45e6-8d38-207bfafddbbc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:38.837903 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 52/96] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:38 2022] GET /identity/v3/role_assignments?user.id=c28c6caabed44035add7bfb1c3325511&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:38.949988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d0cf2-5cd6-429b-afc1-9177f044ece6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:38.971750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:39.004502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0408318-f0cf-4384-90f0-f366b99dc9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:39.005139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0408318-f0cf-4384-90f0-f366b99dc9bb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:39.005436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0408318-f0cf-4384-90f0-f366b99dc9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:39.005751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0408318-f0cf-4384-90f0-f366b99dc9bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:39.006808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0408318-f0cf-4384-90f0-f366b99dc9bb None admin] RBAC: Adding query filter params (role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=94e4dc5051034046b6daf8b6972d78f4, user.id=c28c6caabed44035add7bfb1c3325511) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:39.007201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0408318-f0cf-4384-90f0-f366b99dc9bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:39.008679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0408318-f0cf-4384-90f0-f366b99dc9bb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:39.015612 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 45/97] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:38 2022] GET /identity/v3/role_assignments?user.id=c28c6caabed44035add7bfb1c3325511&scope.project.id=94e4dc5051034046b6daf8b6972d78f4&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:41.539992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4f6c-8bc4-4ba0-b68a-c1b7092e6eb5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:41.541175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4f6c-8bc4-4ba0-b68a-c1b7092e6eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:41.541683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b4f6c-8bc4-4ba0-b68a-c1b7092e6eb5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:41.543238 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 53/98] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:41.549524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa0d56-6a7f-46f1-b953-5a4a9d2d308e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:41.550445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa0d56-6a7f-46f1-b953-5a4a9d2d308e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:41.550966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa0d56-6a7f-46f1-b953-5a4a9d2d308e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:41.574700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c7fa0d56-6a7f-46f1-b953-5a4a9d2d308e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:41.575911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c7fa0d56-6a7f-46f1-b953-5a4a9d2d308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:41.672274 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 46/99] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:41.940556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ce287-cb06-4aba-aeb0-b292bf394731 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:41.941194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ce287-cb06-4aba-aeb0-b292bf394731 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:41.941703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ce287-cb06-4aba-aeb0-b292bf394731 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:41.943238 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 54/100] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:41.955124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064e470-50db-4b86-846e-424495ebfb7a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:41.955706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064e470-50db-4b86-846e-424495ebfb7a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:41.956179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064e470-50db-4b86-846e-424495ebfb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:41.986639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8064e470-50db-4b86-846e-424495ebfb7a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:41.988300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8064e470-50db-4b86-846e-424495ebfb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:41.997682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd41a9-9ad6-4c5d-88ee-50d130e5e36b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:41.997682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd41a9-9ad6-4c5d-88ee-50d130e5e36b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:41.997682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd41a9-9ad6-4c5d-88ee-50d130e5e36b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:41.998938 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 55/101] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:42.006483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe0b38-d98b-4c19-94ca-0fae054e36d1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.007051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe0b38-d98b-4c19-94ca-0fae054e36d1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.007637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe0b38-d98b-4c19-94ca-0fae054e36d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.030455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-60fe0b38-d98b-4c19-94ca-0fae054e36d1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:42.031976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-60fe0b38-d98b-4c19-94ca-0fae054e36d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:42.091853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:42.113008 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 47/102] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:42.116016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c67de-e403-462b-aaf1-da5952704cd7 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.116489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c67de-e403-462b-aaf1-da5952704cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.117566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c67de-e403-462b-aaf1-da5952704cd7 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.118875 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 48/103] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:42.124751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4628d76-4fb6-47fd-98e0-67e67890199c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.125035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4628d76-4fb6-47fd-98e0-67e67890199c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.125311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4628d76-4fb6-47fd-98e0-67e67890199c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.131981 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 56/104] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:42.153228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a4628d76-4fb6-47fd-98e0-67e67890199c None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:42.154506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a4628d76-4fb6-47fd-98e0-67e67890199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:42.171095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe0b38-d98b-4c19-94ca-0fae054e36d1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:42.223972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:42.328044 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 49/105] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:42.363280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bf0d4-b45d-460d-9121-57d066f391ba None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.363588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bf0d4-b45d-460d-9121-57d066f391ba None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.363905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bf0d4-b45d-460d-9121-57d066f391ba None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.365868 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 50/106] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:42.370904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c467b54-fe5d-486e-a449-add54fa4634e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.371192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c467b54-fe5d-486e-a449-add54fa4634e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.371469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c467b54-fe5d-486e-a449-add54fa4634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.400351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe0b38-d98b-4c19-94ca-0fae054e36d1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:42.404007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5c467b54-fe5d-486e-a449-add54fa4634e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:42.405205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5c467b54-fe5d-486e-a449-add54fa4634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:42.451184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341eddd-e0ba-443b-ba7b-b9f898f367b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:42.451923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341eddd-e0ba-443b-ba7b-b9f898f367b3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.452211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341eddd-e0ba-443b-ba7b-b9f898f367b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.452532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341eddd-e0ba-443b-ba7b-b9f898f367b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.453792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341eddd-e0ba-443b-ba7b-b9f898f367b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:42.455667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341eddd-e0ba-443b-ba7b-b9f898f367b3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:42.458120 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 57/107] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:42.460399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ce6b3-8103-42e0-88df-42ea0dcbc4c7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.461013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ce6b3-8103-42e0-88df-42ea0dcbc4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.461434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ce6b3-8103-42e0-88df-42ea0dcbc4c7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.463179 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 58/108] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:42.464065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ce6b3-8103-42e0-88df-42ea0dcbc4c7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:42.525728 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 51/109] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:42.525728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa086-a6f1-4679-be8d-2dd883ece31b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.525728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa086-a6f1-4679-be8d-2dd883ece31b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.525728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fa086-a6f1-4679-be8d-2dd883ece31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.544708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-246fa086-a6f1-4679-be8d-2dd883ece31b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:42.546162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-246fa086-a6f1-4679-be8d-2dd883ece31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:42.547285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:42.639346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ce6b3-8103-42e0-88df-42ea0dcbc4c7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:42.642954 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 52/110] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:42.644272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fa086-a6f1-4679-be8d-2dd883ece31b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:42.677544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:42.680964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056ce2c-d9f8-4048-ac0b-030fa2b7eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:42.681658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056ce2c-d9f8-4048-ac0b-030fa2b7eeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.681956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056ce2c-d9f8-4048-ac0b-030fa2b7eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.682411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056ce2c-d9f8-4048-ac0b-030fa2b7eeb5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.698131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056ce2c-d9f8-4048-ac0b-030fa2b7eeb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-b056ce2c-d9f8-4048-ac0b-030fa2b7eeb5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:42.703419 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:42.704266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:42.705028 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 59/111] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:42.706389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056ce2c-d9f8-4048-ac0b-030fa2b7eeb5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:42.785918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fa086-a6f1-4679-be8d-2dd883ece31b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:42.813842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056ce2c-d9f8-4048-ac0b-030fa2b7eeb5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:42.829451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:42.832793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5bcba-2642-44bd-9e73-5697c466a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:42.833463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5bcba-2642-44bd-9e73-5697c466a826 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.833792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5bcba-2642-44bd-9e73-5697c466a826 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.834130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5bcba-2642-44bd-9e73-5697c466a826 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.835029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5bcba-2642-44bd-9e73-5697c466a826 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:42.836494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5bcba-2642-44bd-9e73-5697c466a826 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:42.851234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e28434-21fa-4e82-af1c-b165df6b7b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:42.851875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28434-21fa-4e82-af1c-b165df6b7b34 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:42.852159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28434-21fa-4e82-af1c-b165df6b7b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:42.852439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28434-21fa-4e82-af1c-b165df6b7b34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:42.863178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e28434-21fa-4e82-af1c-b165df6b7b34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:42.863697 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 53/112] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:21:42 2022] POST /identity/v3/projects => generated 302 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:42.864358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5bcba-2642-44bd-9e73-5697c466a826 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e1e28434-21fa-4e82-af1c-b165df6b7b34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:42.875578 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:42.876510 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:42.877284 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:42.884911 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 60/113] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:21:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:42.884911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e28434-21fa-4e82-af1c-b165df6b7b34 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:42.973867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:43.002348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5bcba-2642-44bd-9e73-5697c466a826 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.008296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e28434-21fa-4e82-af1c-b165df6b7b34 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.045853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c872b-ff90-4685-952a-02eaecb9c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.046771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c872b-ff90-4685-952a-02eaecb9c46d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.047498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c872b-ff90-4685-952a-02eaecb9c46d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.047498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c872b-ff90-4685-952a-02eaecb9c46d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.052352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.052484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.052829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.053598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.054034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:43.055129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.055872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.059143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c872b-ff90-4685-952a-02eaecb9c46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-658c872b-ff90-4685-952a-02eaecb9c46d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:43.064628 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:43.065476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:43.066317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:43.066937 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 61/114] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:43.067015 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 54/115] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:43.067081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.067152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c872b-ff90-4685-952a-02eaecb9c46d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.106850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:43.175888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ababcfa-8a20-460e-875c-43e7e7cd385a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.179682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c872b-ff90-4685-952a-02eaecb9c46d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.213222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47664452-4bbd-45d5-87a1-1cbc643bef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.213893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47664452-4bbd-45d5-87a1-1cbc643bef00 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.214176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47664452-4bbd-45d5-87a1-1cbc643bef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.214451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47664452-4bbd-45d5-87a1-1cbc643bef00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.219014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.220170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.222361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.222361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.222361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:43.222361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.224120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.228666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:43.229547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47664452-4bbd-45d5-87a1-1cbc643bef00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-47664452-4bbd-45d5-87a1-1cbc643bef00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:43.235297 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:43.236274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:43.237066 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 62/116] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:43.242438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47664452-4bbd-45d5-87a1-1cbc643bef00 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.248432 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 55/117] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:43.248432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.343777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:43.350914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47664452-4bbd-45d5-87a1-1cbc643bef00 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.353419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde93de1-c219-4daa-8857-9645629939e5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.386807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99852a19-9b9d-4062-884d-33f0401cbd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.387765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852a19-9b9d-4062-884d-33f0401cbd21 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.388203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852a19-9b9d-4062-884d-33f0401cbd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.388484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852a19-9b9d-4062-884d-33f0401cbd21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.392091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8eced-9889-44a6-9f21-a8f529f0daa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.392756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8eced-9889-44a6-9f21-a8f529f0daa1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.393016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8eced-9889-44a6-9f21-a8f529f0daa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.393289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8eced-9889-44a6-9f21-a8f529f0daa1 None admin] PATH_INFO: `/v3/users/d25ed6bb489a4de3ab0766f724cada1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.404754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99852a19-9b9d-4062-884d-33f0401cbd21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-99852a19-9b9d-4062-884d-33f0401cbd21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:43.407854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:43.408864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 63/118] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:43.409803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99852a19-9b9d-4062-884d-33f0401cbd21 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.418308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8eced-9889-44a6-9f21-a8f529f0daa1 None admin] RBAC: Authorizing `identity:get_user(user_id=d25ed6bb489a4de3ab0766f724cada1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.419792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8eced-9889-44a6-9f21-a8f529f0daa1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.421487 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 56/119] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/users/d25ed6bb489a4de3ab0766f724cada1e => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:43.426674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8eced-9889-44a6-9f21-a8f529f0daa1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.483387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:43.518814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99852a19-9b9d-4062-884d-33f0401cbd21 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.538295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8eced-9889-44a6-9f21-a8f529f0daa1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.557213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.557983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.558277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.558557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.559232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:43.559542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.560948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.570513 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 64/120] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:43.571316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.574880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.575504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.575786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.576061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.576730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:43.577035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.578457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.587990 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 57/121] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:43.587990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.594207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:43.685698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b32de-856b-46c0-87c2-36614fa53ad3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.690810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9f6b-7ee4-4457-a41c-ae86114c33eb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.709005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:43.722802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166274-7d44-4da6-a675-ca9d9536ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.724147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166274-7d44-4da6-a675-ca9d9536ec65 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.724446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166274-7d44-4da6-a675-ca9d9536ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.724806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166274-7d44-4da6-a675-ca9d9536ec65 None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.726015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166274-7d44-4da6-a675-ca9d9536ec65 None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.727501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166274-7d44-4da6-a675-ca9d9536ec65 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.728908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.729360 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 65/122] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:43.730201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166274-7d44-4da6-a675-ca9d9536ec65 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.733956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.734251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.734536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.735215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:43.735645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.737163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.751538 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 58/123] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:43.751538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.831095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:43.848198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166274-7d44-4da6-a675-ca9d9536ec65 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.862734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7ee59-537d-40a2-bc9c-e06b13faedeb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:43.883759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d2ff-c7b5-4778-86dc-e3aae9ad64ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.884363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d2ff-c7b5-4778-86dc-e3aae9ad64ec None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.884644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d2ff-c7b5-4778-86dc-e3aae9ad64ec None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.884915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4d2ff-c7b5-4778-86dc-e3aae9ad64ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.903453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6110b-3d97-45c3-98fd-518a65d0d20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:43.904117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6110b-3d97-45c3-98fd-518a65d0d20e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:43.904388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6110b-3d97-45c3-98fd-518a65d0d20e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:43.904667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6110b-3d97-45c3-98fd-518a65d0d20e None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:43.905854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6110b-3d97-45c3-98fd-518a65d0d20e None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.907306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6110b-3d97-45c3-98fd-518a65d0d20e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.908882 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 59/124] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:43.909653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6110b-3d97-45c3-98fd-518a65d0d20e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:43.954505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4d2ff-c7b5-4778-86dc-e3aae9ad64ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:43.955995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4d2ff-c7b5-4778-86dc-e3aae9ad64ec None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:43.960946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:44.016124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6110b-3d97-45c3-98fd-518a65d0d20e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.043324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:44.055089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c1607-9781-453d-bb61-db49342001b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.055843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1607-9781-453d-bb61-db49342001b9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.056125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1607-9781-453d-bb61-db49342001b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.056514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1607-9781-453d-bb61-db49342001b9 None admin] PATH_INFO: `/v3/users/22586ccbb1964437ab05abad3612640c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.057654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c1607-9781-453d-bb61-db49342001b9 None admin] RBAC: Authorizing `identity:get_user(user_id=22586ccbb1964437ab05abad3612640c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.059110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c1607-9781-453d-bb61-db49342001b9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.059858 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 66/125] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:44.060704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d2ff-c7b5-4778-86dc-e3aae9ad64ec None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.070696 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 60/126] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:43 2022] GET /identity/v3/users/22586ccbb1964437ab05abad3612640c => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:44.070696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c1607-9781-453d-bb61-db49342001b9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.166887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4d2ff-c7b5-4778-86dc-e3aae9ad64ec None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.180164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c1607-9781-453d-bb61-db49342001b9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.197664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:44.203563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726e3ae-ee87-4021-b657-3f37fb19794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.204137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726e3ae-ee87-4021-b657-3f37fb19794d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.204419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726e3ae-ee87-4021-b657-3f37fb19794d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.204693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726e3ae-ee87-4021-b657-3f37fb19794d None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.205834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726e3ae-ee87-4021-b657-3f37fb19794d None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.207299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726e3ae-ee87-4021-b657-3f37fb19794d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.209581 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 67/127] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:44 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:44.210341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726e3ae-ee87-4021-b657-3f37fb19794d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.222719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294281d-cd29-454f-90e8-f995a9102d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.223357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294281d-cd29-454f-90e8-f995a9102d09 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.223638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294281d-cd29-454f-90e8-f995a9102d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.223910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294281d-cd29-454f-90e8-f995a9102d09 None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.225339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294281d-cd29-454f-90e8-f995a9102d09 None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.227201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294281d-cd29-454f-90e8-f995a9102d09 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.228789 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 61/128] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:44 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:44.229382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294281d-cd29-454f-90e8-f995a9102d09 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.269416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:44.322212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726e3ae-ee87-4021-b657-3f37fb19794d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.336254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294281d-cd29-454f-90e8-f995a9102d09 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.355389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ee07b-5464-4f2b-b18b-4e1cc70c84fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.356017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ee07b-5464-4f2b-b18b-4e1cc70c84fb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.356297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ee07b-5464-4f2b-b18b-4e1cc70c84fb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.356570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ee07b-5464-4f2b-b18b-4e1cc70c84fb None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.357716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ee07b-5464-4f2b-b18b-4e1cc70c84fb None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.359161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ee07b-5464-4f2b-b18b-4e1cc70c84fb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.360773 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 68/129] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:44 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:44.361523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ee07b-5464-4f2b-b18b-4e1cc70c84fb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.378204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673e6ab-a980-43f9-8e73-cc2f84019a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.379088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673e6ab-a980-43f9-8e73-cc2f84019a48 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.379576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673e6ab-a980-43f9-8e73-cc2f84019a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.380033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673e6ab-a980-43f9-8e73-cc2f84019a48 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.381769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673e6ab-a980-43f9-8e73-cc2f84019a48 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.383227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673e6ab-a980-43f9-8e73-cc2f84019a48 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.384983 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 62/130] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:44 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:44.390115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673e6ab-a980-43f9-8e73-cc2f84019a48 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.449491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:44.483672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ee07b-5464-4f2b-b18b-4e1cc70c84fb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.500986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673e6ab-a980-43f9-8e73-cc2f84019a48 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.507144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:44.518897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b148a-f400-4b07-b332-972508ea3faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.519537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b148a-f400-4b07-b332-972508ea3faa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.519820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b148a-f400-4b07-b332-972508ea3faa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.520101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b148a-f400-4b07-b332-972508ea3faa None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.521213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b148a-f400-4b07-b332-972508ea3faa None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.527194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b148a-f400-4b07-b332-972508ea3faa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.533938 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 69/131] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:44 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:44.533938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b148a-f400-4b07-b332-972508ea3faa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.546604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e884-3703-4c29-82a1-b466dad855eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.547382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e884-3703-4c29-82a1-b466dad855eb None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.547838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e884-3703-4c29-82a1-b466dad855eb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.548122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e884-3703-4c29-82a1-b466dad855eb None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/users/c28c6caabed44035add7bfb1c3325511/roles/89a7fdb1750447479ee3540579dc1e23` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.550484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129e884-3703-4c29-82a1-b466dad855eb None admin] RBAC: Authorizing `identity:create_grant(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0, user_id=c28c6caabed44035add7bfb1c3325511, role_id=89a7fdb1750447479ee3540579dc1e23)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.552136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129e884-3703-4c29-82a1-b466dad855eb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.567950 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 63/132] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:21:44 2022] PUT /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/users/c28c6caabed44035add7bfb1c3325511/roles/89a7fdb1750447479ee3540579dc1e23 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:21:44.569960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e884-3703-4c29-82a1-b466dad855eb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.671513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b148a-f400-4b07-b332-972508ea3faa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.714923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea0b9b-0003-4648-a56e-e0be9f3c3b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.715672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea0b9b-0003-4648-a56e-e0be9f3c3b10 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.716056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea0b9b-0003-4648-a56e-e0be9f3c3b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.716454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea0b9b-0003-4648-a56e-e0be9f3c3b10 None admin] PATH_INFO: `/v3/projects/94e4dc5051034046b6daf8b6972d78f4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.717872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea0b9b-0003-4648-a56e-e0be9f3c3b10 None admin] RBAC: Authorizing `identity:get_project(project_id=94e4dc5051034046b6daf8b6972d78f4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.719647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea0b9b-0003-4648-a56e-e0be9f3c3b10 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.721597 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 70/133] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:44 2022] GET /identity/v3/projects/94e4dc5051034046b6daf8b6972d78f4 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:44.726896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea0b9b-0003-4648-a56e-e0be9f3c3b10 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.733900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e884-3703-4c29-82a1-b466dad855eb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.752960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:44.785755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3a5cd-d6cd-4aeb-ac46-390092c5f3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.786937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a5cd-d6cd-4aeb-ac46-390092c5f3b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.787915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a5cd-d6cd-4aeb-ac46-390092c5f3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.788428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a5cd-d6cd-4aeb-ac46-390092c5f3b3 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/users/c28c6caabed44035add7bfb1c3325511/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.791048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3a5cd-d6cd-4aeb-ac46-390092c5f3b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0, user_id=c28c6caabed44035add7bfb1c3325511, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.792869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3a5cd-d6cd-4aeb-ac46-390092c5f3b3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.798195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:44.813993 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 64/134] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:21:44 2022] PUT /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/users/c28c6caabed44035add7bfb1c3325511/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:21:44.815146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3a5cd-d6cd-4aeb-ac46-390092c5f3b3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.856125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea0b9b-0003-4648-a56e-e0be9f3c3b10 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:44.910290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:44.912776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:44.913901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:44.915838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:44.916803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] RBAC: Adding query filter params (role.id=2efac5a86ee940c49657b4c67e2d12e6, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0, user.id=d25ed6bb489a4de3ab0766f724cada1e) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:44.917276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:44.920814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:44.932554 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 71/135] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:44 2022] GET /identity/v3/role_assignments?user.id=d25ed6bb489a4de3ab0766f724cada1e&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=2efac5a86ee940c49657b4c67e2d12e6 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:44.934467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:44.995253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3a5cd-d6cd-4aeb-ac46-390092c5f3b3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:45.102995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed745-7399-4275-9d8c-281de61dc443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:45.103950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed745-7399-4275-9d8c-281de61dc443 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:45.104399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed745-7399-4275-9d8c-281de61dc443 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:45.104829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed745-7399-4275-9d8c-281de61dc443 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/users/22586ccbb1964437ab05abad3612640c/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:45.108135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ed745-7399-4275-9d8c-281de61dc443 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0, user_id=22586ccbb1964437ab05abad3612640c, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:45.110613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ed745-7399-4275-9d8c-281de61dc443 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:45.135084 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 65/136] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:21:44 2022] PUT /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/users/22586ccbb1964437ab05abad3612640c/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:21:45.174240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ae0d5-0754-4743-83d3-504107a6b799 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:45.201238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:45.220196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccf21b-80c6-453d-b36c-1ceeb926f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:45.221135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccf21b-80c6-453d-b36c-1ceeb926f79a None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:45.221574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccf21b-80c6-453d-b36c-1ceeb926f79a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:45.222048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccf21b-80c6-453d-b36c-1ceeb926f79a None admin] PATH_INFO: `/v3/projects/94e4dc5051034046b6daf8b6972d78f4/users/c28c6caabed44035add7bfb1c3325511/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:45.224205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccf21b-80c6-453d-b36c-1ceeb926f79a None admin] RBAC: Authorizing `identity:create_grant(project_id=94e4dc5051034046b6daf8b6972d78f4, user_id=c28c6caabed44035add7bfb1c3325511, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:45.226084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccf21b-80c6-453d-b36c-1ceeb926f79a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:45.238287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b5b8b-20c4-4375-98e6-ba3a39e9845e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:45.238736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b5b8b-20c4-4375-98e6-ba3a39e9845e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:45.239111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b5b8b-20c4-4375-98e6-ba3a39e9845e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:45.241266 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 66/137] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:45.247220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93227e34-6bcd-47f6-8e18-75b746b0ed62 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:45.247542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93227e34-6bcd-47f6-8e18-75b746b0ed62 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:45.247921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93227e34-6bcd-47f6-8e18-75b746b0ed62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:45.248793 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 72/138] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:21:44 2022] PUT /identity/v3/projects/94e4dc5051034046b6daf8b6972d78f4/users/c28c6caabed44035add7bfb1c3325511/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:21:45.280126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-93227e34-6bcd-47f6-8e18-75b746b0ed62 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:45.281347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-93227e34-6bcd-47f6-8e18-75b746b0ed62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:45.326707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:45.388083 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 67/139] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:45.798220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccf21b-80c6-453d-b36c-1ceeb926f79a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:45.932052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccf21b-80c6-453d-b36c-1ceeb926f79a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:45.970473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2a4b0-c089-438f-bd50-a5627ddc51d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:45.971105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a4b0-c089-438f-bd50-a5627ddc51d0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:45.971380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a4b0-c089-438f-bd50-a5627ddc51d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:45.971646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a4b0-c089-438f-bd50-a5627ddc51d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:45.972762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2a4b0-c089-438f-bd50-a5627ddc51d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:45.974225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2a4b0-c089-438f-bd50-a5627ddc51d0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:45.976452 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 73/140] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:45.982212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93227e34-6bcd-47f6-8e18-75b746b0ed62 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:46.150239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93227e34-6bcd-47f6-8e18-75b746b0ed62 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:46.203238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb4623-98c3-42f5-abe6-bfc0c85b7438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:46.204001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb4623-98c3-42f5-abe6-bfc0c85b7438 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:46.204287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb4623-98c3-42f5-abe6-bfc0c85b7438 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:46.204565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb4623-98c3-42f5-abe6-bfc0c85b7438 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:46.205225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb4623-98c3-42f5-abe6-bfc0c85b7438 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:46.206684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb4623-98c3-42f5-abe6-bfc0c85b7438 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:46.244198 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 68/141] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:21:45 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:47.570463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a8b73-7b33-40ad-953c-dfef1bc11864 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:47.570979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a8b73-7b33-40ad-953c-dfef1bc11864 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:47.570979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a8b73-7b33-40ad-953c-dfef1bc11864 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:47.572477 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 74/142] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:47.588376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d13c8-876b-4d9e-a2e4-efb44acda836 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:47.588858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d13c8-876b-4d9e-a2e4-efb44acda836 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:47.588950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d13c8-876b-4d9e-a2e4-efb44acda836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:47.613878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6e4d13c8-876b-4d9e-a2e4-efb44acda836 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:47.615148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6e4d13c8-876b-4d9e-a2e4-efb44acda836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:47.634626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:47.658965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f0c99-5083-42f3-b858-f70d8ad8bade None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:47.659139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f0c99-5083-42f3-b858-f70d8ad8bade None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:47.659413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f0c99-5083-42f3-b858-f70d8ad8bade None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:47.660867 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 75/143] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:47.672768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33425267-0db6-4a89-b09f-195ae1349dc9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:47.673055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33425267-0db6-4a89-b09f-195ae1349dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:47.673308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33425267-0db6-4a89-b09f-195ae1349dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:47.704676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-33425267-0db6-4a89-b09f-195ae1349dc9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:47.707641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-33425267-0db6-4a89-b09f-195ae1349dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:47.750010 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 69/144] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:47.791065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:47.822442 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 76/145] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:48.141539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d13c8-876b-4d9e-a2e4-efb44acda836 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:48.264092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d13c8-876b-4d9e-a2e4-efb44acda836 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:48.303683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a22f5-e61d-4ea2-b3c9-4fb8b16623fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:48.304475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a22f5-e61d-4ea2-b3c9-4fb8b16623fc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:48.304755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a22f5-e61d-4ea2-b3c9-4fb8b16623fc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:48.305030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a22f5-e61d-4ea2-b3c9-4fb8b16623fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:48.317021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a22f5-e61d-4ea2-b3c9-4fb8b16623fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-5a2a22f5-e61d-4ea2-b3c9-4fb8b16623fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:48.322230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:48.323176 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:48.323941 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 70/146] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:21:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:48.329187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33425267-0db6-4a89-b09f-195ae1349dc9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:48.331709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e34d84-e142-4152-a6ff-486f6782856c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:48.331992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e34d84-e142-4152-a6ff-486f6782856c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:48.332268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e34d84-e142-4152-a6ff-486f6782856c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:48.334412 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 71/147] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:48.339409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71f9db-9f88-4338-aaba-7d7f7dfce10f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:48.339692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71f9db-9f88-4338-aaba-7d7f7dfce10f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:48.339969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71f9db-9f88-4338-aaba-7d7f7dfce10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:48.367182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fb71f9db-9f88-4338-aaba-7d7f7dfce10f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:48.368377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fb71f9db-9f88-4338-aaba-7d7f7dfce10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:48.370548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:48.480157 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 72/148] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:48.481027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71f9db-9f88-4338-aaba-7d7f7dfce10f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:48.533145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33425267-0db6-4a89-b09f-195ae1349dc9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:48.595208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71f9db-9f88-4338-aaba-7d7f7dfce10f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:48.597738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:48.598828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:48.599273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:48.600241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:48.601069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:48.601534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:48.603110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:48.625531 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 77/149] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:21:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:48.633998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:48.669986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4bdde-8965-41e4-98ca-32fac725f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:48.670888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4bdde-8965-41e4-98ca-32fac725f97c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:48.671407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4bdde-8965-41e4-98ca-32fac725f97c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:48.672717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4bdde-8965-41e4-98ca-32fac725f97c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:48.694252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:48.701081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4bdde-8965-41e4-98ca-32fac725f97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-4ad4bdde-8965-41e4-98ca-32fac725f97c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:48.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:48.709462 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:48.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:48.715822 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 73/150] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:48.722637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4bdde-8965-41e4-98ca-32fac725f97c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:48.748175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:48.857393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e7aa8-e95a-4e70-ba97-f19881de9b51 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:48.946393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d3aa9-9c1e-4c8c-8ccd-1302d9f28e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:48.947328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d3aa9-9c1e-4c8c-8ccd-1302d9f28e88 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:48.947752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d3aa9-9c1e-4c8c-8ccd-1302d9f28e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:48.948175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d3aa9-9c1e-4c8c-8ccd-1302d9f28e88 None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:48.950592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d3aa9-9c1e-4c8c-8ccd-1302d9f28e88 None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:48.951908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4bdde-8965-41e4-98ca-32fac725f97c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:48.952528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d3aa9-9c1e-4c8c-8ccd-1302d9f28e88 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:48.958976 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 78/151] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:48 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:48.968073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d3aa9-9c1e-4c8c-8ccd-1302d9f28e88 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:49.031236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:49.031870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.032165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.032440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:49.033115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:49.033417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:49.034902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:49.051008 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 74/152] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:49.058326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:49.166526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d3aa9-9c1e-4c8c-8ccd-1302d9f28e88 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:49.187447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:49.210889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:49.250384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c466a12-c624-46d2-9d68-e68be4f10ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:49.251387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c466a12-c624-46d2-9d68-e68be4f10ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.251915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c466a12-c624-46d2-9d68-e68be4f10ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.252396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c466a12-c624-46d2-9d68-e68be4f10ce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:49.302689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c17c25-6246-4339-a271-4d294adb83d5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:49.372023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c466a12-c624-46d2-9d68-e68be4f10ce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:49.373525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c466a12-c624-46d2-9d68-e68be4f10ce9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:49.383080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d6946-c571-431d-a530-b98928190d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:49.384035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d6946-c571-431d-a530-b98928190d57 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.384518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d6946-c571-431d-a530-b98928190d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.385015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d6946-c571-431d-a530-b98928190d57 None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:49.390354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d6946-c571-431d-a530-b98928190d57 None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:49.392085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d6946-c571-431d-a530-b98928190d57 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:49.394080 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 75/153] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:49 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:49.397651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d6946-c571-431d-a530-b98928190d57 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:49.579042 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 79/154] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:49.581044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38b45-4c86-4c99-8152-00e11489f1d0 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.581661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38b45-4c86-4c99-8152-00e11489f1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.583596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d6946-c571-431d-a530-b98928190d57 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:49.584667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38b45-4c86-4c99-8152-00e11489f1d0 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:49.587685 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 80/155] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:49.589050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b02e9-ec2d-43b9-8b31-4b73077ab061 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.590306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b02e9-ec2d-43b9-8b31-4b73077ab061 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.590741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b02e9-ec2d-43b9-8b31-4b73077ab061 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:49.596036 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 81/156] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:49.596963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b02e9-ec2d-43b9-8b31-4b73077ab061 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:49.625975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:49.652627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:49.682125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ea835-ee04-4310-a8a3-2cd5a73ed05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:49.682768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea835-ee04-4310-a8a3-2cd5a73ed05e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.683043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea835-ee04-4310-a8a3-2cd5a73ed05e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.683312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ea835-ee04-4310-a8a3-2cd5a73ed05e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:49.704802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ea835-ee04-4310-a8a3-2cd5a73ed05e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-7f4ea835-ee04-4310-a8a3-2cd5a73ed05e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:49.710852 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:49.712237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 76/157] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:49.713091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ea835-ee04-4310-a8a3-2cd5a73ed05e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:49.739297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b02e9-ec2d-43b9-8b31-4b73077ab061 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:49.800768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611832f5-5a47-4e03-9bf0-57645ae60324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:49.801376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611832f5-5a47-4e03-9bf0-57645ae60324 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.801665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611832f5-5a47-4e03-9bf0-57645ae60324 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.801937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611832f5-5a47-4e03-9bf0-57645ae60324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:49.908331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ea835-ee04-4310-a8a3-2cd5a73ed05e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:49.916387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611832f5-5a47-4e03-9bf0-57645ae60324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:49.917887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611832f5-5a47-4e03-9bf0-57645ae60324 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:49.938004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:49.975260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0525668-281b-4328-9b1a-7e50cbe83339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:49.975900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525668-281b-4328-9b1a-7e50cbe83339 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.976178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525668-281b-4328-9b1a-7e50cbe83339 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.976452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525668-281b-4328-9b1a-7e50cbe83339 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:49.978534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0525668-281b-4328-9b1a-7e50cbe83339 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:49.979996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0525668-281b-4328-9b1a-7e50cbe83339 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:49.981606 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 77/158] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:49 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:49.993775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b854a66-34e6-4157-a666-9ada24b52331 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:49.994096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b854a66-34e6-4157-a666-9ada24b52331 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:49.994368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b854a66-34e6-4157-a666-9ada24b52331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.024907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4b854a66-34e6-4157-a666-9ada24b52331 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:50.027386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4b854a66-34e6-4157-a666-9ada24b52331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:50.112457 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 82/159] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:50.113736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb7446-0b21-4344-95e4-86f3e24a70d3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:50.114022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb7446-0b21-4344-95e4-86f3e24a70d3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:50.114352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb7446-0b21-4344-95e4-86f3e24a70d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.129674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:50.143311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dabb7446-0b21-4344-95e4-86f3e24a70d3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:50.144978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dabb7446-0b21-4344-95e4-86f3e24a70d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:50.154188 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 78/160] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:50.155294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b854a66-34e6-4157-a666-9ada24b52331 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:50.268188 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 83/161] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:50.270097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb7446-0b21-4344-95e4-86f3e24a70d3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:50.353490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b854a66-34e6-4157-a666-9ada24b52331 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:50.364545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:50.417813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:50.418442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:50.418717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:50.418984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.419657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:50.419957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:50.421331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:50.438184 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 79/162] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:50.438852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:50.453675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb7446-0b21-4344-95e4-86f3e24a70d3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:50.517855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:50.524598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:50.525147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:50.525423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:50.525710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.526438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] RBAC: Adding query filter params (role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0, user.id=c28c6caabed44035add7bfb1c3325511) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:50.527003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:50.528457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:50.539834 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 84/163] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:50 2022] GET /identity/v3/role_assignments?user.id=c28c6caabed44035add7bfb1c3325511&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:50.540657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:50.625442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fcf99-2b99-49d2-a7c2-388adf5b0537 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:50.714413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817887a1-33cc-4ae3-ba8d-fe932fb019fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:50.715063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817887a1-33cc-4ae3-ba8d-fe932fb019fd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:50.715646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817887a1-33cc-4ae3-ba8d-fe932fb019fd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:50.715929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817887a1-33cc-4ae3-ba8d-fe932fb019fd None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.717148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817887a1-33cc-4ae3-ba8d-fe932fb019fd None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:50.719669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817887a1-33cc-4ae3-ba8d-fe932fb019fd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:50.721480 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 80/164] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:50 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:50.722745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84b388-5d0e-4864-8729-63dee78ea6ab None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:50.723026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84b388-5d0e-4864-8729-63dee78ea6ab None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:50.723299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84b388-5d0e-4864-8729-63dee78ea6ab None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.725234 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 81/165] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:50.730911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84b388-5d0e-4864-8729-63dee78ea6ab None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:50.795663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0206429-1f60-4a75-aa28-4fad6c9ff0d1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:50.799908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:50.861370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787480b-e0a2-4512-805f-55e9633aea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:50.862034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787480b-e0a2-4512-805f-55e9633aea44 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:50.862311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787480b-e0a2-4512-805f-55e9633aea44 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:50.862579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787480b-e0a2-4512-805f-55e9633aea44 None admin] PATH_INFO: `/v3/users/d25ed6bb489a4de3ab0766f724cada1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.863727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787480b-e0a2-4512-805f-55e9633aea44 None admin] RBAC: Authorizing `identity:get_user(user_id=d25ed6bb489a4de3ab0766f724cada1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:50.865288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787480b-e0a2-4512-805f-55e9633aea44 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:50.867055 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 85/166] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:50 2022] GET /identity/v3/users/d25ed6bb489a4de3ab0766f724cada1e => generated 289 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:50.867825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787480b-e0a2-4512-805f-55e9633aea44 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:50.890057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84b388-5d0e-4864-8729-63dee78ea6ab None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:50.914567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:50.940754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e0767-d86f-4e4f-ad83-381c40da3cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:50.941408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e0767-d86f-4e4f-ad83-381c40da3cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:50.941722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e0767-d86f-4e4f-ad83-381c40da3cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:50.941994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e0767-d86f-4e4f-ad83-381c40da3cb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.956835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e0767-d86f-4e4f-ad83-381c40da3cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-f79e0767-d86f-4e4f-ad83-381c40da3cb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:50.963670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:50.964519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:50.965503 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 82/167] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:21:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:50.966176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b057545-8e5f-4207-86f7-d9c638878d1b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:50.966268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b057545-8e5f-4207-86f7-d9c638878d1b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:50.966839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b057545-8e5f-4207-86f7-d9c638878d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:50.986714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8b057545-8e5f-4207-86f7-d9c638878d1b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:50.988150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8b057545-8e5f-4207-86f7-d9c638878d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:51.002510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787480b-e0a2-4512-805f-55e9633aea44 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.047409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.047946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.048227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.048500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.049497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] RBAC: Adding query filter params (role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0, user.id=c28c6caabed44035add7bfb1c3325511) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:51.049849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:51.052093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:51.060410 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 86/168] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:50 2022] GET /identity/v3/role_assignments?user.id=c28c6caabed44035add7bfb1c3325511&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:51.061769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.073564 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 83/169] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:51.074764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b057545-8e5f-4207-86f7-d9c638878d1b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.123466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:51.242915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951417a-74aa-4528-b69e-5cece2264f53 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.282463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2633a37-204b-43b7-9364-0e272a6d6def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.283366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2633a37-204b-43b7-9364-0e272a6d6def None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.283643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2633a37-204b-43b7-9364-0e272a6d6def None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.283917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2633a37-204b-43b7-9364-0e272a6d6def None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.284657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b057545-8e5f-4207-86f7-d9c638878d1b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.293957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:51.296338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2633a37-204b-43b7-9364-0e272a6d6def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d2633a37-204b-43b7-9364-0e272a6d6def None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:51.304047 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:51.304896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:51.305694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:51.306368 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 87/170] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:51.306667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2633a37-204b-43b7-9364-0e272a6d6def None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.332768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbf23a-09e5-4ca8-a630-bebf609b1c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.334198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbf23a-09e5-4ca8-a630-bebf609b1c9d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.334589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbf23a-09e5-4ca8-a630-bebf609b1c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.334960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbf23a-09e5-4ca8-a630-bebf609b1c9d None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.336215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbf23a-09e5-4ca8-a630-bebf609b1c9d None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:51.337818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbf23a-09e5-4ca8-a630-bebf609b1c9d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:51.339527 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 84/171] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:51 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:51.340730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e230a-e23e-4e6a-94bf-38d4c6721011 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.341012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e230a-e23e-4e6a-94bf-38d4c6721011 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.341282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e230a-e23e-4e6a-94bf-38d4c6721011 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.342700 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 85/172] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:51.343389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e230a-e23e-4e6a-94bf-38d4c6721011 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.436661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:51.445517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2633a37-204b-43b7-9364-0e272a6d6def None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.489931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e230a-e23e-4e6a-94bf-38d4c6721011 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.490336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba6484-7f17-49f4-8aed-250519d4d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.491060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba6484-7f17-49f4-8aed-250519d4d52a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.491487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba6484-7f17-49f4-8aed-250519d4d52a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.491779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba6484-7f17-49f4-8aed-250519d4d52a None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.493270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba6484-7f17-49f4-8aed-250519d4d52a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:51.493579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba6484-7f17-49f4-8aed-250519d4d52a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:51.495003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba6484-7f17-49f4-8aed-250519d4d52a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:51.504990 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 88/173] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:51.508219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804937d-d985-49e1-bc39-83aee132cbf9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.508711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804937d-d985-49e1-bc39-83aee132cbf9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.509086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804937d-d985-49e1-bc39-83aee132cbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.527769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5804937d-d985-49e1-bc39-83aee132cbf9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:51.527983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.528600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.528878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.528977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5804937d-d985-49e1-bc39-83aee132cbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:51.529151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.529839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:51.530142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:51.532446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:51.541310 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 86/174] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:51.542102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.575614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:51.598997 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 89/175] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:51.600471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804937d-d985-49e1-bc39-83aee132cbf9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.650132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0442491-c402-4e2f-9cff-883ddb1db2ed None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.683370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894d1b7-6592-4fe6-aaf8-321a5ae9a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.684033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894d1b7-6592-4fe6-aaf8-321a5ae9a308 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.684311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894d1b7-6592-4fe6-aaf8-321a5ae9a308 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.684579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894d1b7-6592-4fe6-aaf8-321a5ae9a308 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/users/d25ed6bb489a4de3ab0766f724cada1e/roles/2efac5a86ee940c49657b4c67e2d12e6` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.686540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894d1b7-6592-4fe6-aaf8-321a5ae9a308 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0, user_id=d25ed6bb489a4de3ab0766f724cada1e, role_id=2efac5a86ee940c49657b4c67e2d12e6)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:51.687976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894d1b7-6592-4fe6-aaf8-321a5ae9a308 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:51.696955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:51.704269 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 87/176] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:21:51 2022] PUT /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/users/d25ed6bb489a4de3ab0766f724cada1e/roles/2efac5a86ee940c49657b4c67e2d12e6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:21:51.705305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894d1b7-6592-4fe6-aaf8-321a5ae9a308 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.719847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804937d-d985-49e1-bc39-83aee132cbf9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.768520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ad293-ee43-47f1-ae56-002e86b83592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.769832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad293-ee43-47f1-ae56-002e86b83592 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.770319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad293-ee43-47f1-ae56-002e86b83592 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.770599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad293-ee43-47f1-ae56-002e86b83592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.788183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ad293-ee43-47f1-ae56-002e86b83592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-048ad293-ee43-47f1-ae56-002e86b83592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:51.795777 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:51.802083 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 90/177] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:51.803767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef8004-b24a-4033-b288-4445501f69dc None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.804439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef8004-b24a-4033-b288-4445501f69dc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.804439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef8004-b24a-4033-b288-4445501f69dc None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.804439 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 91/178] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:51.804439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef8004-b24a-4033-b288-4445501f69dc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.852177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:51.862024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894d1b7-6592-4fe6-aaf8-321a5ae9a308 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.911676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8f787-fafc-4dbc-ae6c-a8a51a1c7967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.912972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8f787-fafc-4dbc-ae6c-a8a51a1c7967 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.913248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8f787-fafc-4dbc-ae6c-a8a51a1c7967 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.913519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8f787-fafc-4dbc-ae6c-a8a51a1c7967 None admin] PATH_INFO: `/v3/users/22586ccbb1964437ab05abad3612640c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.914810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8f787-fafc-4dbc-ae6c-a8a51a1c7967 None admin] RBAC: Authorizing `identity:get_user(user_id=22586ccbb1964437ab05abad3612640c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:51.916257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8f787-fafc-4dbc-ae6c-a8a51a1c7967 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:51.917879 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 88/179] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:51 2022] GET /identity/v3/users/22586ccbb1964437ab05abad3612640c => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:51.918635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8f787-fafc-4dbc-ae6c-a8a51a1c7967 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:51.925677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef8004-b24a-4033-b288-4445501f69dc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:51.966002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31905d04-b1d3-4f77-bc34-15077be1f337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:51.966701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31905d04-b1d3-4f77-bc34-15077be1f337 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.967393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31905d04-b1d3-4f77-bc34-15077be1f337 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.967679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31905d04-b1d3-4f77-bc34-15077be1f337 None admin] PATH_INFO: `/v3/users/c28c6caabed44035add7bfb1c3325511` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:51.969147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31905d04-b1d3-4f77-bc34-15077be1f337 None admin] RBAC: Authorizing `identity:get_user(user_id=c28c6caabed44035add7bfb1c3325511)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:51.971750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31905d04-b1d3-4f77-bc34-15077be1f337 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:51.974372 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 92/180] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:51 2022] GET /identity/v3/users/c28c6caabed44035add7bfb1c3325511 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:51.979806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90d0b1-3802-40ed-a70f-35d76cf4b886 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:51.979806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90d0b1-3802-40ed-a70f-35d76cf4b886 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:51.979806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90d0b1-3802-40ed-a70f-35d76cf4b886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.001677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8e90d0b1-3802-40ed-a70f-35d76cf4b886 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:52.003394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8e90d0b1-3802-40ed-a70f-35d76cf4b886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:52.019044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:52.028534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8f787-fafc-4dbc-ae6c-a8a51a1c7967 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.064512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.065132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.065464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.065772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.066491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:52.066793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:52.068251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:52.077650 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 89/181] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:52.078280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:52.093668 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 93/182] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:52.093668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90d0b1-3802-40ed-a70f-35d76cf4b886 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:52.120542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:52.202590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90d0b1-3802-40ed-a70f-35d76cf4b886 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.220541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492536da-b31c-493d-8192-eb3f295f08a2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.238214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7815d-f795-45e8-80fc-53bc09dfde93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.239091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7815d-f795-45e8-80fc-53bc09dfde93 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.239376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7815d-f795-45e8-80fc-53bc09dfde93 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.239646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7815d-f795-45e8-80fc-53bc09dfde93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.262046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbc19e-4886-4f85-9a13-ca903fcc9442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.262983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc19e-4886-4f85-9a13-ca903fcc9442 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.263266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc19e-4886-4f85-9a13-ca903fcc9442 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.263541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbc19e-4886-4f85-9a13-ca903fcc9442 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.264669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbc19e-4886-4f85-9a13-ca903fcc9442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:52.266026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:52.266628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbc19e-4886-4f85-9a13-ca903fcc9442 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:52.269291 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 90/183] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:52.270285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbc19e-4886-4f85-9a13-ca903fcc9442 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:52.323179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7815d-f795-45e8-80fc-53bc09dfde93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:52.324747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7815d-f795-45e8-80fc-53bc09dfde93 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:52.374741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:52.382595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbc19e-4886-4f85-9a13-ca903fcc9442 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.422341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d9c6-c231-431c-8b01-f00277fd53df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.422952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d9c6-c231-431c-8b01-f00277fd53df None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.423225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d9c6-c231-431c-8b01-f00277fd53df None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.423494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d9c6-c231-431c-8b01-f00277fd53df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.457186 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 94/184] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:52.458611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7815d-f795-45e8-80fc-53bc09dfde93 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:52.496708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82d9c6-c231-431c-8b01-f00277fd53df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:52.498290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82d9c6-c231-431c-8b01-f00277fd53df None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:52.511317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:52.564075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7815d-f795-45e8-80fc-53bc09dfde93 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.597819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038ee4-2237-425f-84b9-a8a3737375a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.598450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038ee4-2237-425f-84b9-a8a3737375a6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.598728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038ee4-2237-425f-84b9-a8a3737375a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.598997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038ee4-2237-425f-84b9-a8a3737375a6 None admin] PATH_INFO: `/v3/users/72a8ea0c360a4ba680056f7b7f1802fb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.605145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:52.615748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23038ee4-2237-425f-84b9-a8a3737375a6 None admin] RBAC: Authorizing `identity:get_user(user_id=72a8ea0c360a4ba680056f7b7f1802fb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:52.617392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23038ee4-2237-425f-84b9-a8a3737375a6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:52.617714 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 91/185] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:52.618858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d9c6-c231-431c-8b01-f00277fd53df None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:52.619397 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 95/186] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:52 2022] GET /identity/v3/users/72a8ea0c360a4ba680056f7b7f1802fb => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:52.620287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038ee4-2237-425f-84b9-a8a3737375a6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:52.726648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d9c6-c231-431c-8b01-f00277fd53df None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.741708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038ee4-2237-425f-84b9-a8a3737375a6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.744371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:52.765983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf897-4114-4016-9e75-8f38784f54a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.766602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf897-4114-4016-9e75-8f38784f54a3 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.766880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf897-4114-4016-9e75-8f38784f54a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.767155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacf897-4114-4016-9e75-8f38784f54a3 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.767796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacf897-4114-4016-9e75-8f38784f54a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:52.769308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacf897-4114-4016-9e75-8f38784f54a3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:52.778619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea93d27-f47c-479b-861d-cdcb4733f24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.779463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea93d27-f47c-479b-861d-cdcb4733f24f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.779940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea93d27-f47c-479b-861d-cdcb4733f24f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.780357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea93d27-f47c-479b-861d-cdcb4733f24f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.791882 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 92/187] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:21:52 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:52.792704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf897-4114-4016-9e75-8f38784f54a3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:52.801562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea93d27-f47c-479b-861d-cdcb4733f24f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-bea93d27-f47c-479b-861d-cdcb4733f24f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:52.810517 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:52.811509 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:21:52.812582 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 96/188] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:21:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:52.813342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea93d27-f47c-479b-861d-cdcb4733f24f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:52.871012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:52.939144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea93d27-f47c-479b-861d-cdcb4733f24f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.945284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacf897-4114-4016-9e75-8f38784f54a3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:52.986338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac978ee6-233c-4136-a95e-6e0220f908d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.986522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd47a4d-4320-4647-b4c9-5cf1989633ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:52.987055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd47a4d-4320-4647-b4c9-5cf1989633ed None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.987414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd47a4d-4320-4647-b4c9-5cf1989633ed None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.987707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd47a4d-4320-4647-b4c9-5cf1989633ed None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.988076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac978ee6-233c-4136-a95e-6e0220f908d7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:52.988550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac978ee6-233c-4136-a95e-6e0220f908d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:52.988866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac978ee6-233c-4136-a95e-6e0220f908d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:52.988974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd47a4d-4320-4647-b4c9-5cf1989633ed None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:52.991242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd47a4d-4320-4647-b4c9-5cf1989633ed None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:52.994080 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 93/189] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:52 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:52.994888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd47a4d-4320-4647-b4c9-5cf1989633ed None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:53.027741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:53.076543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac978ee6-233c-4136-a95e-6e0220f908d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:53.078070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac978ee6-233c-4136-a95e-6e0220f908d7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:53.135809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd47a4d-4320-4647-b4c9-5cf1989633ed None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:53.163777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:53.180423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b93c38-1825-4d1c-82b8-dba0cc88ecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:53.181076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93c38-1825-4d1c-82b8-dba0cc88ecbc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:53.181354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93c38-1825-4d1c-82b8-dba0cc88ecbc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:53.181702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93c38-1825-4d1c-82b8-dba0cc88ecbc None admin] PATH_INFO: `/v3/projects/94e4dc5051034046b6daf8b6972d78f4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:53.182802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b93c38-1825-4d1c-82b8-dba0cc88ecbc None admin] RBAC: Authorizing `identity:get_project(project_id=94e4dc5051034046b6daf8b6972d78f4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:53.184240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b93c38-1825-4d1c-82b8-dba0cc88ecbc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:53.186108 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 94/190] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:52 2022] GET /identity/v3/projects/94e4dc5051034046b6daf8b6972d78f4 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:53.186660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b93c38-1825-4d1c-82b8-dba0cc88ecbc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:53.216863 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 97/191] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:53.217748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac978ee6-233c-4136-a95e-6e0220f908d7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:53.302862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b93c38-1825-4d1c-82b8-dba0cc88ecbc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:53.306875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:53.342280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:53.342698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:53.342973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:53.343248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:53.343910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:53.344206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:53.345582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:53.354560 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 95/192] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:21:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:53.355456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:53.362724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac978ee6-233c-4136-a95e-6e0220f908d7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:53.412954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:53.413863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:53.414150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:53.414426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:53.415222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:53.415815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0, user.id=22586ccbb1964437ab05abad3612640c) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:53.417238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:53.418935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:53.428631 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 98/193] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:53 2022] GET /identity/v3/role_assignments?user.id=22586ccbb1964437ab05abad3612640c&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:53.433876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:53.511930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2b009-b43f-4d0c-aabd-07a2749dd2fb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:53.603399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:53.603947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:53.604225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:53.604497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:53.605313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] RBAC: Adding query filter params (role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=94e4dc5051034046b6daf8b6972d78f4, user.id=c28c6caabed44035add7bfb1c3325511) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:53.605640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:53.607136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:53.623975 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 96/194] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:53 2022] GET /identity/v3/role_assignments?user.id=c28c6caabed44035add7bfb1c3325511&scope.project.id=94e4dc5051034046b6daf8b6972d78f4&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:53.624837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:53.679047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216dd7-8fdf-4e93-8fe1-133e3a9f6415 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:53.727985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:53.747425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f355dc3-4ad5-498f-a286-503cd2039d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:53.748064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f355dc3-4ad5-498f-a286-503cd2039d68 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:53.748338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f355dc3-4ad5-498f-a286-503cd2039d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:53.748606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f355dc3-4ad5-498f-a286-503cd2039d68 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:53.750309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f355dc3-4ad5-498f-a286-503cd2039d68 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:53.751837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f355dc3-4ad5-498f-a286-503cd2039d68 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:53.753717 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 99/195] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:53 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:53.761778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f355dc3-4ad5-498f-a286-503cd2039d68 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:53.829712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb81460-9523-44a9-86d5-942aaa60bb05 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:53.854392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:53.894722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a054d1-9c4b-485d-8a1a-9ddabc185b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:53.895384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a054d1-9c4b-485d-8a1a-9ddabc185b8a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:53.895644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a054d1-9c4b-485d-8a1a-9ddabc185b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:53.895916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a054d1-9c4b-485d-8a1a-9ddabc185b8a None admin] PATH_INFO: `/v3/users/72a8ea0c360a4ba680056f7b7f1802fb` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:53.902505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a054d1-9c4b-485d-8a1a-9ddabc185b8a None admin] RBAC: Authorizing `identity:get_user(user_id=72a8ea0c360a4ba680056f7b7f1802fb)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:53.903986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a054d1-9c4b-485d-8a1a-9ddabc185b8a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:53.907883 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 97/196] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:53 2022] GET /identity/v3/users/72a8ea0c360a4ba680056f7b7f1802fb => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:53.911521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a054d1-9c4b-485d-8a1a-9ddabc185b8a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:53.955540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f355dc3-4ad5-498f-a286-503cd2039d68 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:54.022826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2c4de-54dc-425a-b382-2417d6bdc0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:54.023511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2c4de-54dc-425a-b382-2417d6bdc0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:54.023791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2c4de-54dc-425a-b382-2417d6bdc0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:54.024113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2c4de-54dc-425a-b382-2417d6bdc0f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:54.024984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2c4de-54dc-425a-b382-2417d6bdc0f1 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user.id=72a8ea0c360a4ba680056f7b7f1802fb) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:54.025293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2c4de-54dc-425a-b382-2417d6bdc0f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:54.027206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2c4de-54dc-425a-b382-2417d6bdc0f1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:54.039140 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 100/197] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:53 2022] GET /identity/v3/role_assignments?user.id=72a8ea0c360a4ba680056f7b7f1802fb&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:54.083095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a054d1-9c4b-485d-8a1a-9ddabc185b8a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:54.129505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1da03-1256-42ec-988c-ab9eba66fe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:54.130185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1da03-1256-42ec-988c-ab9eba66fe9b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:54.130525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1da03-1256-42ec-988c-ab9eba66fe9b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:54.130806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1da03-1256-42ec-988c-ab9eba66fe9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:54.193593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:54.221005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1da03-1256-42ec-988c-ab9eba66fe9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:54.222539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1da03-1256-42ec-988c-ab9eba66fe9b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:54.294678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfef14-1ed1-49cd-ba68-262476a92f33 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:54.295017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfef14-1ed1-49cd-ba68-262476a92f33 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:54.295219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfef14-1ed1-49cd-ba68-262476a92f33 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:54.297523 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 101/198] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:54.307716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a5628-f302-4578-b225-7e0dc76f64e5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:54.308038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a5628-f302-4578-b225-7e0dc76f64e5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:54.308271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a5628-f302-4578-b225-7e0dc76f64e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:54.331477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-353a5628-f302-4578-b225-7e0dc76f64e5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:54.332695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-353a5628-f302-4578-b225-7e0dc76f64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:54.352312 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 98/199] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:54.361872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1da03-1256-42ec-988c-ab9eba66fe9b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:54.429509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:54.452336 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 102/200] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:54.493749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:54.524747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1da03-1256-42ec-988c-ab9eba66fe9b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:54.576195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1298f-b8cb-40b1-9625-2a1b500d6de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:54.576840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1298f-b8cb-40b1-9625-2a1b500d6de8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:54.577122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1298f-b8cb-40b1-9625-2a1b500d6de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:54.577396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1298f-b8cb-40b1-9625-2a1b500d6de8 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:54.578623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1298f-b8cb-40b1-9625-2a1b500d6de8 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:54.580079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1298f-b8cb-40b1-9625-2a1b500d6de8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:54.581931 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 99/201] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:54 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:54.588098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a5628-f302-4578-b225-7e0dc76f64e5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:54.707322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a5628-f302-4578-b225-7e0dc76f64e5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:54.750629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1298f-b8cb-40b1-9625-2a1b500d6de8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:54.765654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:54.765654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:54.765654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:54.765654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:54.766074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] RBAC: Adding query filter params (role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user.id=72a8ea0c360a4ba680056f7b7f1802fb) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:54.766074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:54.766074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:54.767841 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 103/202] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:54 2022] GET /identity/v3/role_assignments?user.id=72a8ea0c360a4ba680056f7b7f1802fb&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:54.919787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1298f-b8cb-40b1-9625-2a1b500d6de8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:54.966073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd445731-024c-4198-a84f-8bfee6e678be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:54.966695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd445731-024c-4198-a84f-8bfee6e678be None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:54.966978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd445731-024c-4198-a84f-8bfee6e678be None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:54.967251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd445731-024c-4198-a84f-8bfee6e678be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:54.986652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd445731-024c-4198-a84f-8bfee6e678be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-cd445731-024c-4198-a84f-8bfee6e678be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:54.992918 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:54.993905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:21:54.994699 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:54.996085 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 100/203] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:21:55.002384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:55.074049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:55.126887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309439f8-fc42-482d-90a6-f30c582f653d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:55.172283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:55.172898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:55.173180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:55.173454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:55.174213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:55.174523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:55.176030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:55.187402 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 104/204] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:55.193733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd445731-024c-4198-a84f-8bfee6e678be None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:55.205523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:55.317680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd445731-024c-4198-a84f-8bfee6e678be None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:55.360048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427946fd-1cd5-45e2-a95b-8e5d3fd56ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:55.360691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427946fd-1cd5-45e2-a95b-8e5d3fd56ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:55.360972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427946fd-1cd5-45e2-a95b-8e5d3fd56ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:55.361248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427946fd-1cd5-45e2-a95b-8e5d3fd56ce3 None admin] PATH_INFO: `/v3/users/d25ed6bb489a4de3ab0766f724cada1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:55.362438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427946fd-1cd5-45e2-a95b-8e5d3fd56ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=d25ed6bb489a4de3ab0766f724cada1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:55.363957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427946fd-1cd5-45e2-a95b-8e5d3fd56ce3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:55.365977 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 101/205] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:55 2022] GET /identity/v3/users/d25ed6bb489a4de3ab0766f724cada1e => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:55.371910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:55.508205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3146c6-f58d-4eba-9e41-5d51b9db65a7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:55.528114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:55.554530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d69e-0cac-42c1-9873-4dfbdc33b454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:55.555151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d69e-0cac-42c1-9873-4dfbdc33b454 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:55.555432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d69e-0cac-42c1-9873-4dfbdc33b454 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:55.555703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d69e-0cac-42c1-9873-4dfbdc33b454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:55.633959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076d69e-0cac-42c1-9873-4dfbdc33b454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:55.635465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076d69e-0cac-42c1-9873-4dfbdc33b454 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:55.765063 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 105/206] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:55.770708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427946fd-1cd5-45e2-a95b-8e5d3fd56ce3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:55.858420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:55.884507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427946fd-1cd5-45e2-a95b-8e5d3fd56ce3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:55.924843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb3ad-cfe2-4e4d-87ee-1c8198501e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:55.925591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb3ad-cfe2-4e4d-87ee-1c8198501e12 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:55.925795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb3ad-cfe2-4e4d-87ee-1c8198501e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:55.926072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb3ad-cfe2-4e4d-87ee-1c8198501e12 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:55.930329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6fb3ad-cfe2-4e4d-87ee-1c8198501e12 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:55.931809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6fb3ad-cfe2-4e4d-87ee-1c8198501e12 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:55.933499 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 102/207] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:55 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:55.939480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d69e-0cac-42c1-9873-4dfbdc33b454 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:55.961290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:56.073445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d69e-0cac-42c1-9873-4dfbdc33b454 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:56.087972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13e454-4960-4a26-bbfc-e4e379e192af None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.088254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13e454-4960-4a26-bbfc-e4e379e192af None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.088529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13e454-4960-4a26-bbfc-e4e379e192af None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.090094 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 103/208] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:56.099035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eeab3-505d-4a73-b76a-c5ff37d453a1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.099926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eeab3-505d-4a73-b76a-c5ff37d453a1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.100203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eeab3-505d-4a73-b76a-c5ff37d453a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.123977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a73eeab3-505d-4a73-b76a-c5ff37d453a1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:56.125175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a73eeab3-505d-4a73-b76a-c5ff37d453a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:56.133876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f3ca8-26ad-40f1-8da8-ea675ff9e340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:56.134967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f3ca8-26ad-40f1-8da8-ea675ff9e340 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.135194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f3ca8-26ad-40f1-8da8-ea675ff9e340 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.135466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f3ca8-26ad-40f1-8da8-ea675ff9e340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.136196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f3ca8-26ad-40f1-8da8-ea675ff9e340 None admin] RBAC: Adding query filter params (role.id=2efac5a86ee940c49657b4c67e2d12e6, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0, user.id=d25ed6bb489a4de3ab0766f724cada1e) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:56.136857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f3ca8-26ad-40f1-8da8-ea675ff9e340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:56.138674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f3ca8-26ad-40f1-8da8-ea675ff9e340 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:56.146479 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 106/209] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:55 2022] GET /identity/v3/role_assignments?user.id=d25ed6bb489a4de3ab0766f724cada1e&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=2efac5a86ee940c49657b4c67e2d12e6 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:56.147664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8b09f-3b5e-43df-8e25-cde9b62911a1 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.147946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8b09f-3b5e-43df-8e25-cde9b62911a1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.148215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8b09f-3b5e-43df-8e25-cde9b62911a1 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.149696 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 107/210] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:56.156666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b0714-ce4c-4df8-ae8e-fa2d6ec1f1da None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.157026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b0714-ce4c-4df8-ae8e-fa2d6ec1f1da None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.157350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b0714-ce4c-4df8-ae8e-fa2d6ec1f1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.186500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fb2b0714-ce4c-4df8-ae8e-fa2d6ec1f1da None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:56.189579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fb2b0714-ce4c-4df8-ae8e-fa2d6ec1f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:56.268397 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 104/211] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:56.308956 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 108/212] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:56.577502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eeab3-505d-4a73-b76a-c5ff37d453a1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:56.612840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b0714-ce4c-4df8-ae8e-fa2d6ec1f1da None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:56.624380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:56.667712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:56.695746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eeab3-505d-4a73-b76a-c5ff37d453a1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:56.727917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b0714-ce4c-4df8-ae8e-fa2d6ec1f1da None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:56.735512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd2f62-be44-4364-9154-c7ed1a9e5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:56.736155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd2f62-be44-4364-9154-c7ed1a9e5475 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.736434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd2f62-be44-4364-9154-c7ed1a9e5475 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.736706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd2f62-be44-4364-9154-c7ed1a9e5475 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.747964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd2f62-be44-4364-9154-c7ed1a9e5475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-79dd2f62-be44-4364-9154-c7ed1a9e5475 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:56.756074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:56.757248 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:56.758148 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 105/213] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:21:56.766304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd2f62-be44-4364-9154-c7ed1a9e5475 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:56.780617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57232ea-996a-4a3b-b957-abf642752893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:56.781889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57232ea-996a-4a3b-b957-abf642752893 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.782323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57232ea-996a-4a3b-b957-abf642752893 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.782741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57232ea-996a-4a3b-b957-abf642752893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.784626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57232ea-996a-4a3b-b957-abf642752893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:56.786436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57232ea-996a-4a3b-b957-abf642752893 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:56.789451 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 109/214] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:21:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:56.794936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57232ea-996a-4a3b-b957-abf642752893 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:56.901175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:56.901364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd2f62-be44-4364-9154-c7ed1a9e5475 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:56.931581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57232ea-996a-4a3b-b957-abf642752893 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:56.947210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:56.948447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.948928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.949339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.950181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:56.950646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:56.952367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:56.953942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:56.969206 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 106/215] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:21:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:56.970739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:56.972961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f9af6-f64b-4897-aedb-3f1b6b4cb5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:56.973860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9af6-f64b-4897-aedb-3f1b6b4cb5ee None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:56.974426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9af6-f64b-4897-aedb-3f1b6b4cb5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:56.974967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f9af6-f64b-4897-aedb-3f1b6b4cb5ee None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:56.975828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f9af6-f64b-4897-aedb-3f1b6b4cb5ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:56.977652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f9af6-f64b-4897-aedb-3f1b6b4cb5ee None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:57.012223 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 110/216] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:21:56 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:21:57.019304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76306ce-5f59-4663-a0cc-8f17848d9edd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.019597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76306ce-5f59-4663-a0cc-8f17848d9edd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.019881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76306ce-5f59-4663-a0cc-8f17848d9edd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.021583 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 111/217] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:57.027584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e769e1-427a-4fc8-a065-fcaeea456e4a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.028097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e769e1-427a-4fc8-a065-fcaeea456e4a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.028519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e769e1-427a-4fc8-a065-fcaeea456e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.050693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-82e769e1-427a-4fc8-a065-fcaeea456e4a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:57.052808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-82e769e1-427a-4fc8-a065-fcaeea456e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:57.094811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf081031-2ab4-4c76-bba2-561b7339d269 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:57.132587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af75bca-0ea3-4e53-839e-d4cb904426c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:57.133488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af75bca-0ea3-4e53-839e-d4cb904426c9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.133938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af75bca-0ea3-4e53-839e-d4cb904426c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.134358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af75bca-0ea3-4e53-839e-d4cb904426c9 None admin] PATH_INFO: `/v3/users/cc1888a6562e46d0a17a64cc3ce55e9d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.146700 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 112/218] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:21:57.150539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af75bca-0ea3-4e53-839e-d4cb904426c9 None admin] RBAC: Authorizing `identity:get_user(user_id=cc1888a6562e46d0a17a64cc3ce55e9d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:57.152309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af75bca-0ea3-4e53-839e-d4cb904426c9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:57.154383 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 107/219] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:56 2022] GET /identity/v3/users/cc1888a6562e46d0a17a64cc3ce55e9d => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:57.159732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e769e1-427a-4fc8-a065-fcaeea456e4a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:57.233531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:57.278283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e769e1-427a-4fc8-a065-fcaeea456e4a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:57.314538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7880c-01e4-48f4-8c13-07d3f98f6639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:57.315307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7880c-01e4-48f4-8c13-07d3f98f6639 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.315545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7880c-01e4-48f4-8c13-07d3f98f6639 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.315861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7880c-01e4-48f4-8c13-07d3f98f6639 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.395761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7880c-01e4-48f4-8c13-07d3f98f6639 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:57.395761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7880c-01e4-48f4-8c13-07d3f98f6639 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:57.435323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af75bca-0ea3-4e53-839e-d4cb904426c9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:57.505097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:57.523862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:57.551718 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 113/220] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:21:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:21:57.557644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7880c-01e4-48f4-8c13-07d3f98f6639 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:57.591123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af75bca-0ea3-4e53-839e-d4cb904426c9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:57.637663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979559e1-117c-438d-ae96-6c0532bbdaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:57.638310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979559e1-117c-438d-ae96-6c0532bbdaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.639112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979559e1-117c-438d-ae96-6c0532bbdaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.639386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979559e1-117c-438d-ae96-6c0532bbdaa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.655897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979559e1-117c-438d-ae96-6c0532bbdaa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-979559e1-117c-438d-ae96-6c0532bbdaa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:57.660823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:57.662298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:57.663085 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 108/221] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:21:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:21:57.664443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f232bf-d5e4-4c91-8a43-a1806b0dfc52 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.664443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f232bf-d5e4-4c91-8a43-a1806b0dfc52 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.664443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f232bf-d5e4-4c91-8a43-a1806b0dfc52 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.665732 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 109/222] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:57.670534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f232bf-d5e4-4c91-8a43-a1806b0dfc52 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:57.674500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7880c-01e4-48f4-8c13-07d3f98f6639 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:57.722746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886cef8-ce2b-47a8-b860-dfa53e76116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:57.724342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886cef8-ce2b-47a8-b860-dfa53e76116d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.725116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886cef8-ce2b-47a8-b860-dfa53e76116d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.725585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886cef8-ce2b-47a8-b860-dfa53e76116d None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.727591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886cef8-ce2b-47a8-b860-dfa53e76116d None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:57.729345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886cef8-ce2b-47a8-b860-dfa53e76116d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:57.732083 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 114/223] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:57 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:57.733445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97540a8c-3974-4bd7-a3fc-02abe427f1d9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.733967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97540a8c-3974-4bd7-a3fc-02abe427f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.734431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97540a8c-3974-4bd7-a3fc-02abe427f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.759062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-97540a8c-3974-4bd7-a3fc-02abe427f1d9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:57.760629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-97540a8c-3974-4bd7-a3fc-02abe427f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:57.788569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f232bf-d5e4-4c91-8a43-a1806b0dfc52 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:57.792159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:57.805881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:57.827266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:57.827940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.828221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.828492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.829159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:57.829458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:57.831075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:57.840219 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 110/224] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:21:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:57.841248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:57.848000 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 115/225] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:57.849423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97540a8c-3974-4bd7-a3fc-02abe427f1d9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:57.954351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97540a8c-3974-4bd7-a3fc-02abe427f1d9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:57.992666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fdc6a-db62-43a5-b8f0-4f9f6a82eaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:57.993400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fdc6a-db62-43a5-b8f0-4f9f6a82eaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:57.993786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fdc6a-db62-43a5-b8f0-4f9f6a82eaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:57.994217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fdc6a-db62-43a5-b8f0-4f9f6a82eaf5 None admin] PATH_INFO: `/v3/users/72a8ea0c360a4ba680056f7b7f1802fb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:57.995419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fdc6a-db62-43a5-b8f0-4f9f6a82eaf5 None admin] RBAC: Authorizing `identity:get_user(user_id=72a8ea0c360a4ba680056f7b7f1802fb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:57.997030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fdc6a-db62-43a5-b8f0-4f9f6a82eaf5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:57.999033 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 116/226] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:57 2022] GET /identity/v3/users/72a8ea0c360a4ba680056f7b7f1802fb => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:58.004443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fdc6a-db62-43a5-b8f0-4f9f6a82eaf5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:58.011374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e40ca8-03d9-41b7-b5d8-04ddf6ebba12 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:58.069981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:58.091271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:58.091886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:58.092228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:58.092497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:58.093270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] RBAC: Adding query filter params (role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user.id=cc1888a6562e46d0a17a64cc3ce55e9d) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:58.093579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:58.096445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:58.115369 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 111/227] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:21:57 2022] GET /identity/v3/role_assignments?user.id=cc1888a6562e46d0a17a64cc3ce55e9d&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:58.180602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fdc6a-db62-43a5-b8f0-4f9f6a82eaf5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:58.204960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:58.227567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d1b6d-a8ad-4ad3-94b7-b2734eda91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:58.228203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d1b6d-a8ad-4ad3-94b7-b2734eda91d2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:58.228480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d1b6d-a8ad-4ad3-94b7-b2734eda91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:58.228811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d1b6d-a8ad-4ad3-94b7-b2734eda91d2 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:58.237654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d1b6d-a8ad-4ad3-94b7-b2734eda91d2 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:58.237654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d1b6d-a8ad-4ad3-94b7-b2734eda91d2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:58.237654 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 117/228] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:58 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:21:58.243546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d1b6d-a8ad-4ad3-94b7-b2734eda91d2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:58.326082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:58.410157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e04e08-e320-4a4d-9d54-e8fa5dca07b9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:58.421023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:58.425907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d1b6d-a8ad-4ad3-94b7-b2734eda91d2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:58.447094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44411b5e-aed5-4742-b49a-9d0f8ac4c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:58.447742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44411b5e-aed5-4742-b49a-9d0f8ac4c20d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:58.448058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44411b5e-aed5-4742-b49a-9d0f8ac4c20d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:58.448333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44411b5e-aed5-4742-b49a-9d0f8ac4c20d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:58.459570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44411b5e-aed5-4742-b49a-9d0f8ac4c20d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-44411b5e-aed5-4742-b49a-9d0f8ac4c20d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:21:58.465121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:21:58.466031 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:21:58.466833 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 112/229] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:21:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:21:58.473807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44411b5e-aed5-4742-b49a-9d0f8ac4c20d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:58.474866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99dc03-7eca-4b48-9e69-315ef232b447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:58.475784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99dc03-7eca-4b48-9e69-315ef232b447 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:58.476274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99dc03-7eca-4b48-9e69-315ef232b447 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:58.476748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99dc03-7eca-4b48-9e69-315ef232b447 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/users/72a8ea0c360a4ba680056f7b7f1802fb/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:58.479293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99dc03-7eca-4b48-9e69-315ef232b447 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user_id=72a8ea0c360a4ba680056f7b7f1802fb, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:58.481050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99dc03-7eca-4b48-9e69-315ef232b447 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:58.509180 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 118/230] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:21:58 2022] PUT /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/users/72a8ea0c360a4ba680056f7b7f1802fb/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:21:58.600856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:58.604153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44411b5e-aed5-4742-b49a-9d0f8ac4c20d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:58.642700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:58.643727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:58.644206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:58.644689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:58.645556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:21:58.646670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:58.648516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:58.658837 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 113/231] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:21:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:58.665887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99dc03-7eca-4b48-9e69-315ef232b447 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:58.790421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99dc03-7eca-4b48-9e69-315ef232b447 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:58.828686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a717955-4a10-4f59-ac28-cce50ba84561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:58.829724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a717955-4a10-4f59-ac28-cce50ba84561 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:58.830204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a717955-4a10-4f59-ac28-cce50ba84561 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:58.830672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a717955-4a10-4f59-ac28-cce50ba84561 None admin] PATH_INFO: `/v3/users/72a8ea0c360a4ba680056f7b7f1802fb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:58.832015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a717955-4a10-4f59-ac28-cce50ba84561 None admin] RBAC: Authorizing `identity:get_user(user_id=72a8ea0c360a4ba680056f7b7f1802fb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:58.834309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a717955-4a10-4f59-ac28-cce50ba84561 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:58.836510 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 119/232] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:21:58 2022] GET /identity/v3/users/72a8ea0c360a4ba680056f7b7f1802fb => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:58.842258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:58.948210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80df2f-76c0-443f-b228-17730f49d39c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:58.985060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95d89b-4bb9-477a-b769-8f9e97807821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:58.986096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d89b-4bb9-477a-b769-8f9e97807821 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:58.986674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d89b-4bb9-477a-b769-8f9e97807821 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:58.987218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95d89b-4bb9-477a-b769-8f9e97807821 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:58.988738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95d89b-4bb9-477a-b769-8f9e97807821 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:58.990958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95d89b-4bb9-477a-b769-8f9e97807821 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:58.993069 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 114/233] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:21:58 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:21:59.006011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a717955-4a10-4f59-ac28-cce50ba84561 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:21:59.036240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:59.149094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a717955-4a10-4f59-ac28-cce50ba84561 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:21:59.190400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230abc4-a342-4b14-8ac5-020455757c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:21:59.191976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230abc4-a342-4b14-8ac5-020455757c9a None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:59.192490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230abc4-a342-4b14-8ac5-020455757c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:59.193508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230abc4-a342-4b14-8ac5-020455757c9a None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/users/72a8ea0c360a4ba680056f7b7f1802fb/roles/89a7fdb1750447479ee3540579dc1e23` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:59.198101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230abc4-a342-4b14-8ac5-020455757c9a None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user_id=72a8ea0c360a4ba680056f7b7f1802fb, role_id=89a7fdb1750447479ee3540579dc1e23)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:21:59.200011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230abc4-a342-4b14-8ac5-020455757c9a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:21:59.232363 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 120/234] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:21:59 2022] PUT /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/users/72a8ea0c360a4ba680056f7b7f1802fb/roles/89a7fdb1750447479ee3540579dc1e23 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:21:59.655451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ee744-d287-4253-a726-0f33606a579f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:59.655965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ee744-d287-4253-a726-0f33606a579f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:59.656040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ee744-d287-4253-a726-0f33606a579f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:59.657593 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 115/235] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:21:59.663167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10568c5b-ce64-4aef-bbf4-16fd4b16c146 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:59.663449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10568c5b-ce64-4aef-bbf4-16fd4b16c146 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:59.663718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10568c5b-ce64-4aef-bbf4-16fd4b16c146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:59.690617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-10568c5b-ce64-4aef-bbf4-16fd4b16c146 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:59.691808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-10568c5b-ce64-4aef-bbf4-16fd4b16c146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:59.790014 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 121/236] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:21:59.851702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8cca1-af07-45d1-a9b8-d3763e908382 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:59.852329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8cca1-af07-45d1-a9b8-d3763e908382 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:59.852804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8cca1-af07-45d1-a9b8-d3763e908382 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:59.854451 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 116/237] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:21:59.860565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b69e1-9378-47fa-8da4-b1d62eac91ca None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:21:59.860889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b69e1-9378-47fa-8da4-b1d62eac91ca None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:21:59.861121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b69e1-9378-47fa-8da4-b1d62eac91ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:21:59.887030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e17b69e1-9378-47fa-8da4-b1d62eac91ca None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:21:59.888207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e17b69e1-9378-47fa-8da4-b1d62eac91ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:21:59.903558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:21:59.973837 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 122/238] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:00.128898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8cca1-af07-45d1-a9b8-d3763e908382 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:00.172774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:00.267317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8cca1-af07-45d1-a9b8-d3763e908382 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:00.307868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8cb2c-054c-4c05-a898-788eef6525d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:00.308570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8cb2c-054c-4c05-a898-788eef6525d9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:00.308915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8cb2c-054c-4c05-a898-788eef6525d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:00.309246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8cb2c-054c-4c05-a898-788eef6525d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:00.310390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8cb2c-054c-4c05-a898-788eef6525d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:00.311955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8cb2c-054c-4c05-a898-788eef6525d9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:00.313540 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 117/239] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:00.319664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b69e1-9378-47fa-8da4-b1d62eac91ca None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:00.323039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8cb2c-054c-4c05-a898-788eef6525d9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:00.501768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b69e1-9378-47fa-8da4-b1d62eac91ca None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:00.506028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:00.514782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8cb2c-054c-4c05-a898-788eef6525d9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:00.568798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f71a1-f4e5-473b-9f88-7210a95d780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:00.569333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f71a1-f4e5-473b-9f88-7210a95d780b None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:00.569617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f71a1-f4e5-473b-9f88-7210a95d780b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:00.569919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f71a1-f4e5-473b-9f88-7210a95d780b None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:00.570573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f71a1-f4e5-473b-9f88-7210a95d780b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:00.574417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f71a1-f4e5-473b-9f88-7210a95d780b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:00.576562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2f151-f8bb-4bd1-845a-6ffe3d3b66e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:00.577209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2f151-f8bb-4bd1-845a-6ffe3d3b66e4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:00.577488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2f151-f8bb-4bd1-845a-6ffe3d3b66e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:00.577788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2f151-f8bb-4bd1-845a-6ffe3d3b66e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:00.595091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2f151-f8bb-4bd1-845a-6ffe3d3b66e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-13d2f151-f8bb-4bd1-845a-6ffe3d3b66e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:00.604661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:00.605522 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:00.606338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 118/240] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:00.611858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2f151-f8bb-4bd1-845a-6ffe3d3b66e4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:00.633645 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 123/241] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:22:00 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:22:00.740197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2f151-f8bb-4bd1-845a-6ffe3d3b66e4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:00.777983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:00.778617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:00.778901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:00.779219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:00.779904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:00.780209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:00.781622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:00.790866 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 119/242] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:00.796856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f71a1-f4e5-473b-9f88-7210a95d780b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:00.804931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:00.930581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f71a1-f4e5-473b-9f88-7210a95d780b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:00.976903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b709b-a22c-40b8-a4bd-0aadfb29dfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:00.977864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b709b-a22c-40b8-a4bd-0aadfb29dfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:00.978372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b709b-a22c-40b8-a4bd-0aadfb29dfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:00.978843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b709b-a22c-40b8-a4bd-0aadfb29dfe7 None admin] PATH_INFO: `/v3/users/7b3945bdedbc4362a2a9a1f55599a164` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:00.994862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b709b-a22c-40b8-a4bd-0aadfb29dfe7 None admin] RBAC: Authorizing `identity:get_user(user_id=7b3945bdedbc4362a2a9a1f55599a164)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:00.998319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b709b-a22c-40b8-a4bd-0aadfb29dfe7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:01.001372 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 124/243] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:00 2022] GET /identity/v3/users/7b3945bdedbc4362a2a9a1f55599a164 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:01.009931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:01.040763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:01.130742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48e591-f2d3-41a3-9c20-c33aa84a33ec None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:01.169317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9df22-46b7-4ff9-a4d4-11d94469d02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:01.169970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9df22-46b7-4ff9-a4d4-11d94469d02b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:01.170247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9df22-46b7-4ff9-a4d4-11d94469d02b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:01.170519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9df22-46b7-4ff9-a4d4-11d94469d02b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:01.201085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae84f0d-c30a-4768-9bea-9eb4e24a277d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:01.201715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae84f0d-c30a-4768-9bea-9eb4e24a277d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:01.202261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae84f0d-c30a-4768-9bea-9eb4e24a277d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:01.203759 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 125/244] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:01.210573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f0f0-8bfe-4dc3-ba08-bfb486c26825 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:01.210859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f0f0-8bfe-4dc3-ba08-bfb486c26825 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:01.211137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f0f0-8bfe-4dc3-ba08-bfb486c26825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:01.236084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1a97f0f0-8bfe-4dc3-ba08-bfb486c26825 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:01.237790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1a97f0f0-8bfe-4dc3-ba08-bfb486c26825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:01.257690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9df22-46b7-4ff9-a4d4-11d94469d02b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:01.257690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9df22-46b7-4ff9-a4d4-11d94469d02b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:01.274215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:01.331240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:01.363213 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 126/245] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:01.371807 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 120/246] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:01.378198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97f0f0-8bfe-4dc3-ba08-bfb486c26825 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:01.486879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787b1f0-23fd-4cfb-a6b0-4d210ed777ac None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:01.487483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787b1f0-23fd-4cfb-a6b0-4d210ed777ac None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:01.487935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787b1f0-23fd-4cfb-a6b0-4d210ed777ac None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:01.489468 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 121/247] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:22:01.496161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0cfe-4cee-48c5-85b0-d21b0fce8daf None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:01.496682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0cfe-4cee-48c5-85b0-d21b0fce8daf None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:01.497127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c0cfe-4cee-48c5-85b0-d21b0fce8daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:01.518460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-866c0cfe-4cee-48c5-85b0-d21b0fce8daf None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:01.523065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-866c0cfe-4cee-48c5-85b0-d21b0fce8daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:01.528547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97f0f0-8bfe-4dc3-ba08-bfb486c26825 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:01.594651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efb49a-3445-47cd-b90f-791b27977924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:01.595292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efb49a-3445-47cd-b90f-791b27977924 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:01.595569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efb49a-3445-47cd-b90f-791b27977924 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:01.595841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efb49a-3445-47cd-b90f-791b27977924 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:01.597770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efb49a-3445-47cd-b90f-791b27977924 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:01.599387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efb49a-3445-47cd-b90f-791b27977924 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:01.601331 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 127/248] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:01 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:01.611990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efb49a-3445-47cd-b90f-791b27977924 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:01.638401 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 122/249] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:01.684730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:01.730182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efb49a-3445-47cd-b90f-791b27977924 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:01.760796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c0cfe-4cee-48c5-85b0-d21b0fce8daf None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:01.776188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:01.777087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:01.777369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:01.777662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:01.778394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] RBAC: Adding query filter params (role.id=2efac5a86ee940c49657b4c67e2d12e6, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user.id=7b3945bdedbc4362a2a9a1f55599a164) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:01.778704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:01.780167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:01.790199 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 128/250] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:01 2022] GET /identity/v3/role_assignments?user.id=7b3945bdedbc4362a2a9a1f55599a164&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=2efac5a86ee940c49657b4c67e2d12e6 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:01.850075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:01.930018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c0cfe-4cee-48c5-85b0-d21b0fce8daf None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:01.978950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1b71c-8900-459c-a892-0b0000098f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:01.980068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b71c-8900-459c-a892-0b0000098f36 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:01.980541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b71c-8900-459c-a892-0b0000098f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:01.981002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b71c-8900-459c-a892-0b0000098f36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:01.995039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1b71c-8900-459c-a892-0b0000098f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-72d1b71c-8900-459c-a892-0b0000098f36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:02.000221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:02.001170 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:02.002424 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 123/251] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:02.009890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:02.039684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1b71c-8900-459c-a892-0b0000098f36 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:02.214053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1b71c-8900-459c-a892-0b0000098f36 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:02.226003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:02.233499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8daee5-4030-4841-a0aa-aff86f0850d5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:02.264996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:02.278882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6012a0-8b5f-4665-9757-15995c351d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:02.279827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6012a0-8b5f-4665-9757-15995c351d4c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:02.280300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6012a0-8b5f-4665-9757-15995c351d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:02.280788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6012a0-8b5f-4665-9757-15995c351d4c None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:02.281748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6012a0-8b5f-4665-9757-15995c351d4c None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:02.282378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6012a0-8b5f-4665-9757-15995c351d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:02.283920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a5b65-c1e0-48c2-8396-bbec1cde1efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:02.284305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6012a0-8b5f-4665-9757-15995c351d4c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:02.284550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5b65-c1e0-48c2-8396-bbec1cde1efd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:02.284828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5b65-c1e0-48c2-8396-bbec1cde1efd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:02.285099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a5b65-c1e0-48c2-8396-bbec1cde1efd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:02.295962 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 129/252] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:02.297574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0c941-d2e3-49fa-a5ca-00e5df10e6ed None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:02.298208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0c941-d2e3-49fa-a5ca-00e5df10e6ed None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:02.298893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0c941-d2e3-49fa-a5ca-00e5df10e6ed None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:02.300642 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 130/253] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:02.306379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0c941-d2e3-49fa-a5ca-00e5df10e6ed None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:02.319812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a5b65-c1e0-48c2-8396-bbec1cde1efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-980a5b65-c1e0-48c2-8396-bbec1cde1efd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:02.325327 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:02.326213 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:02.326998 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:02.333786 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 124/254] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:22:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:02.333786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0486509-abee-4f52-8894-641fc802ddc2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:02.333786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0486509-abee-4f52-8894-641fc802ddc2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:02.333786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0486509-abee-4f52-8894-641fc802ddc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:02.369868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c0486509-abee-4f52-8894-641fc802ddc2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:02.371439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c0486509-abee-4f52-8894-641fc802ddc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:02.460258 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 125/255] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:02.461125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0486509-abee-4f52-8894-641fc802ddc2 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:02.492600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0c941-d2e3-49fa-a5ca-00e5df10e6ed None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:02.538763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83845e6b-a6b3-4380-a1eb-da4eaa9ae564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:02.539390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83845e6b-a6b3-4380-a1eb-da4eaa9ae564 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:02.539674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83845e6b-a6b3-4380-a1eb-da4eaa9ae564 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:02.540489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83845e6b-a6b3-4380-a1eb-da4eaa9ae564 None admin] PATH_INFO: `/v3/users/cc1888a6562e46d0a17a64cc3ce55e9d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:02.541665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83845e6b-a6b3-4380-a1eb-da4eaa9ae564 None admin] RBAC: Authorizing `identity:get_user(user_id=cc1888a6562e46d0a17a64cc3ce55e9d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:02.544029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83845e6b-a6b3-4380-a1eb-da4eaa9ae564 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:02.546570 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 131/256] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:02 2022] GET /identity/v3/users/cc1888a6562e46d0a17a64cc3ce55e9d => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:02.555253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83845e6b-a6b3-4380-a1eb-da4eaa9ae564 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:02.675923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:02.686021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:02.688403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0486509-abee-4f52-8894-641fc802ddc2 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:02.764798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:02.765462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:02.765768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:02.766042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:02.767797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:02.768101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:02.769482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:02.788776 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 126/257] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:02.798775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:02.804451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83845e6b-a6b3-4380-a1eb-da4eaa9ae564 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:02.881655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262af5a-0764-4b33-b725-443eddeb0283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:02.881655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262af5a-0764-4b33-b725-443eddeb0283 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:02.881655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262af5a-0764-4b33-b725-443eddeb0283 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:02.881655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262af5a-0764-4b33-b725-443eddeb0283 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:02.883973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262af5a-0764-4b33-b725-443eddeb0283 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:02.885548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262af5a-0764-4b33-b725-443eddeb0283 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:02.888672 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 132/258] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:02 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:02.893652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262af5a-0764-4b33-b725-443eddeb0283 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:02.954556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84412335-8ee5-4834-b2d0-0392b4d37b25 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:02.991249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e91c67-f91c-4412-91ac-7097fee04979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:02.992122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e91c67-f91c-4412-91ac-7097fee04979 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:02.992641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e91c67-f91c-4412-91ac-7097fee04979 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:02.993052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e91c67-f91c-4412-91ac-7097fee04979 None admin] PATH_INFO: `/v3/users/72a8ea0c360a4ba680056f7b7f1802fb` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:02.994542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e91c67-f91c-4412-91ac-7097fee04979 None admin] RBAC: Authorizing `identity:get_user(user_id=72a8ea0c360a4ba680056f7b7f1802fb)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:02.996135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e91c67-f91c-4412-91ac-7097fee04979 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:02.998196 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 127/259] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:02 2022] GET /identity/v3/users/72a8ea0c360a4ba680056f7b7f1802fb => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:02.999183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e91c67-f91c-4412-91ac-7097fee04979 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:03.050165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:03.131473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262af5a-0764-4b33-b725-443eddeb0283 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:03.154500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e91c67-f91c-4412-91ac-7097fee04979 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:03.168969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:03.194220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ad2e6-96d1-4b41-bbb4-e88b02d5d6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:03.194880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ad2e6-96d1-4b41-bbb4-e88b02d5d6c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:03.195159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ad2e6-96d1-4b41-bbb4-e88b02d5d6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:03.195429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ad2e6-96d1-4b41-bbb4-e88b02d5d6c6 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/users/cc1888a6562e46d0a17a64cc3ce55e9d/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:03.198443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ad2e6-96d1-4b41-bbb4-e88b02d5d6c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user_id=cc1888a6562e46d0a17a64cc3ce55e9d, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:03.199899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ad2e6-96d1-4b41-bbb4-e88b02d5d6c6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:03.217389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076eae7-05a0-4282-b667-43e5cd5bedd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:03.218121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076eae7-05a0-4282-b667-43e5cd5bedd9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:03.218400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076eae7-05a0-4282-b667-43e5cd5bedd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:03.218673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076eae7-05a0-4282-b667-43e5cd5bedd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:03.219359 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 133/260] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:22:02 2022] PUT /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/users/cc1888a6562e46d0a17a64cc3ce55e9d/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:22:03.220161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ad2e6-96d1-4b41-bbb4-e88b02d5d6c6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:03.243601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076eae7-05a0-4282-b667-43e5cd5bedd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1076eae7-05a0-4282-b667-43e5cd5bedd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:03.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:03.249467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:03.250298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:03.255752 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 128/261] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:22:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:03.256687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076eae7-05a0-4282-b667-43e5cd5bedd9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:03.423217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ad2e6-96d1-4b41-bbb4-e88b02d5d6c6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:03.448681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:03.466779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076eae7-05a0-4282-b667-43e5cd5bedd9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:03.494935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b71d3-eb5e-492c-af25-a951e24e879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:03.495578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b71d3-eb5e-492c-af25-a951e24e879b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:03.495832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b71d3-eb5e-492c-af25-a951e24e879b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:03.496103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b71d3-eb5e-492c-af25-a951e24e879b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:03.545071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:03.545717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:03.545997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:03.546266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:03.546937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:03.547243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:03.548634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:03.563687 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 129/262] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:03.570594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:03.602043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:03.618575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b71d3-eb5e-492c-af25-a951e24e879b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:03.620189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b71d3-eb5e-492c-af25-a951e24e879b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:03.764286 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 134/263] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:03.769669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b71d3-eb5e-492c-af25-a951e24e879b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:03.777334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6b7e4-c2d7-45b4-af82-a02dc59cf953 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:03.837778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17ecae-ae05-4dd9-a994-6fa21badf223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:03.838419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17ecae-ae05-4dd9-a994-6fa21badf223 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:03.838694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17ecae-ae05-4dd9-a994-6fa21badf223 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:03.838965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17ecae-ae05-4dd9-a994-6fa21badf223 None admin] PATH_INFO: `/v3/users/72a8ea0c360a4ba680056f7b7f1802fb` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:03.840090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17ecae-ae05-4dd9-a994-6fa21badf223 None admin] RBAC: Authorizing `identity:get_user(user_id=72a8ea0c360a4ba680056f7b7f1802fb)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:03.841540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17ecae-ae05-4dd9-a994-6fa21badf223 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:03.843148 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 130/264] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:03 2022] GET /identity/v3/users/72a8ea0c360a4ba680056f7b7f1802fb => generated 283 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:03.848494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17ecae-ae05-4dd9-a994-6fa21badf223 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:03.882924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:03.937995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:03.948904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17ecae-ae05-4dd9-a994-6fa21badf223 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:03.969394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b71d3-eb5e-492c-af25-a951e24e879b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:03.983496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdecf32-f799-43f7-9c9f-50b6b1373eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:03.984109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdecf32-f799-43f7-9c9f-50b6b1373eec None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:03.984386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdecf32-f799-43f7-9c9f-50b6b1373eec None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:03.984658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdecf32-f799-43f7-9c9f-50b6b1373eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.055034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c27d5-5a74-4249-9250-715729fd5224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:04.055448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdecf32-f799-43f7-9c9f-50b6b1373eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:04.056228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c27d5-5a74-4249-9250-715729fd5224 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.056722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c27d5-5a74-4249-9250-715729fd5224 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.056921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdecf32-f799-43f7-9c9f-50b6b1373eec None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:04.057367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c27d5-5a74-4249-9250-715729fd5224 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.062331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c27d5-5a74-4249-9250-715729fd5224 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:04.064010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c27d5-5a74-4249-9250-715729fd5224 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:04.065940 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 135/265] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:03 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:04.072151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c27d5-5a74-4249-9250-715729fd5224 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:04.085615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:04.196404 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 131/266] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 09:22:04.205698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdecf32-f799-43f7-9c9f-50b6b1373eec None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:04.306202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c27d5-5a74-4249-9250-715729fd5224 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:04.341803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6be16-859a-4d72-a994-60e239f4687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:04.342414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6be16-859a-4d72-a994-60e239f4687a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.342699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6be16-859a-4d72-a994-60e239f4687a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.342974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6be16-859a-4d72-a994-60e239f4687a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.343881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6be16-859a-4d72-a994-60e239f4687a None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user.id=72a8ea0c360a4ba680056f7b7f1802fb) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:04.344196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6be16-859a-4d72-a994-60e239f4687a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:04.345680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6be16-859a-4d72-a994-60e239f4687a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:04.348056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdecf32-f799-43f7-9c9f-50b6b1373eec None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:04.352611 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 136/267] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:04 2022] GET /identity/v3/role_assignments?user.id=72a8ea0c360a4ba680056f7b7f1802fb&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:04.354066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4e6df-c047-4233-aeb0-21ba71f26039 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.354355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4e6df-c047-4233-aeb0-21ba71f26039 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.356305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4e6df-c047-4233-aeb0-21ba71f26039 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.357782 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 137/268] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:04.359741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89aeea-2b74-4417-973a-564c49f86bdc None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.360023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89aeea-2b74-4417-973a-564c49f86bdc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.360296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89aeea-2b74-4417-973a-564c49f86bdc None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.371248 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 138/269] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:04.372671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19471-b2d9-4849-96bb-8070f9bc5332 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.372968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19471-b2d9-4849-96bb-8070f9bc5332 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.373248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19471-b2d9-4849-96bb-8070f9bc5332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.399477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-33b19471-b2d9-4849-96bb-8070f9bc5332 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:04.400730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-33b19471-b2d9-4849-96bb-8070f9bc5332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:04.406214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:04.419474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:04.438182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bf01a-3d3d-43e0-a7d9-b03db4c8aa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:04.438841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf01a-3d3d-43e0-a7d9-b03db4c8aa21 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.439123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf01a-3d3d-43e0-a7d9-b03db4c8aa21 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.439390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf01a-3d3d-43e0-a7d9-b03db4c8aa21 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.442439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bf01a-3d3d-43e0-a7d9-b03db4c8aa21 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:04.444670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bf01a-3d3d-43e0-a7d9-b03db4c8aa21 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:04.446731 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 132/270] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:04 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:04.447911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42d24a-3372-49cb-a9cf-468fcfae278a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.448192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42d24a-3372-49cb-a9cf-468fcfae278a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.448463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42d24a-3372-49cb-a9cf-468fcfae278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.488472 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 139/271] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:04.489357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b19471-b2d9-4849-96bb-8070f9bc5332 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:04.491063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4a42d24a-3372-49cb-a9cf-468fcfae278a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:04.492394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4a42d24a-3372-49cb-a9cf-468fcfae278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:04.610002 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 133/272] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:04.669994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b19471-b2d9-4849-96bb-8070f9bc5332 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:04.722377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:04.723346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.723871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.724337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.725254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] RBAC: Adding query filter params (role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user.id=72a8ea0c360a4ba680056f7b7f1802fb) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:04.725798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:04.727442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:04.738017 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 140/273] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:04 2022] GET /identity/v3/role_assignments?user.id=72a8ea0c360a4ba680056f7b7f1802fb&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:04.925669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbca57-2330-4628-85e5-6dc664358598 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.926183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbca57-2330-4628-85e5-6dc664358598 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.926183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbca57-2330-4628-85e5-6dc664358598 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.927647 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 134/274] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:04.932070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:04.934542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f04c2-61e5-454f-a9c0-d72e5a048a95 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:04.934898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f04c2-61e5-454f-a9c0-d72e5a048a95 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:04.935294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f04c2-61e5-454f-a9c0-d72e5a048a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:04.942223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:04.964481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f88f04c2-61e5-454f-a9c0-d72e5a048a95 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:04.965751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f88f04c2-61e5-454f-a9c0-d72e5a048a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:05.045030 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 135/275] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:05.048077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f04c2-61e5-454f-a9c0-d72e5a048a95 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:05.057874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8de12-6c30-4af0-a794-0410649bad87 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:05.099688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0535b-dd6a-40c8-894d-636dae266b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:05.100329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0535b-dd6a-40c8-894d-636dae266b3b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:05.100608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0535b-dd6a-40c8-894d-636dae266b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:05.100883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0535b-dd6a-40c8-894d-636dae266b3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:05.113290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:05.121456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0535b-dd6a-40c8-894d-636dae266b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e5c0535b-dd6a-40c8-894d-636dae266b3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:05.127500 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:05.128396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:05.129596 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 141/276] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:05.135815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0535b-dd6a-40c8-894d-636dae266b3b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:05.182555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f04c2-61e5-454f-a9c0-d72e5a048a95 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:05.216063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:05.229739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0851a01-257d-496f-a2e2-5896bf6668e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:05.230723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0851a01-257d-496f-a2e2-5896bf6668e1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:05.231846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0851a01-257d-496f-a2e2-5896bf6668e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:05.232314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0851a01-257d-496f-a2e2-5896bf6668e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:05.234321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0851a01-257d-496f-a2e2-5896bf6668e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:05.236273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0851a01-257d-496f-a2e2-5896bf6668e1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:05.238628 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 136/277] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:05.245531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0851a01-257d-496f-a2e2-5896bf6668e1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:05.263065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0535b-dd6a-40c8-894d-636dae266b3b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:05.309128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:05.309779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:05.310060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:05.310334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:05.311125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:05.311432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:05.312824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:05.325926 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 142/278] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:05.331684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:05.376901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0851a01-257d-496f-a2e2-5896bf6668e1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:05.401791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:05.427561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee760-46a8-428a-8be6-4bf458d605b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:05.428172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee760-46a8-428a-8be6-4bf458d605b1 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:05.428448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee760-46a8-428a-8be6-4bf458d605b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:05.429116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee760-46a8-428a-8be6-4bf458d605b1 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:05.429789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deee760-46a8-428a-8be6-4bf458d605b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:05.431562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deee760-46a8-428a-8be6-4bf458d605b1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:05.454608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58c30-a466-4c6b-bbea-5028521cf7ea None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:05.461309 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 137/279] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:22:05 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:22:05.461309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee760-46a8-428a-8be6-4bf458d605b1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:05.491574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:05.508118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3726b06-8b5f-4a2a-bdeb-5f3bb47773c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:05.508771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3726b06-8b5f-4a2a-bdeb-5f3bb47773c2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:05.509053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3726b06-8b5f-4a2a-bdeb-5f3bb47773c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:05.510135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3726b06-8b5f-4a2a-bdeb-5f3bb47773c2 None admin] PATH_INFO: `/v3/users/af049965a3514d7dac8cbdbc69963392` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:05.535485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3726b06-8b5f-4a2a-bdeb-5f3bb47773c2 None admin] RBAC: Authorizing `identity:get_user(user_id=af049965a3514d7dac8cbdbc69963392)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:05.538947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3726b06-8b5f-4a2a-bdeb-5f3bb47773c2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:05.542268 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 143/280] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:05 2022] GET /identity/v3/users/af049965a3514d7dac8cbdbc69963392 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:05.548236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3726b06-8b5f-4a2a-bdeb-5f3bb47773c2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:05.606067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee760-46a8-428a-8be6-4bf458d605b1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:05.657575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aeaaec-add0-4a6b-b5ed-fe256d89f025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:05.657575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeaaec-add0-4a6b-b5ed-fe256d89f025 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:05.657575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeaaec-add0-4a6b-b5ed-fe256d89f025 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:05.657575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeaaec-add0-4a6b-b5ed-fe256d89f025 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:05.673768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aeaaec-add0-4a6b-b5ed-fe256d89f025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-49aeaaec-add0-4a6b-b5ed-fe256d89f025 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:05.679404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:05.680307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:05.681106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:05.681733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3726b06-8b5f-4a2a-bdeb-5f3bb47773c2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:05.682137 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 138/281] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:05.687860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aeaaec-add0-4a6b-b5ed-fe256d89f025 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:05.727862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:05.728896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:05.729390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:05.729758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:05.730503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] RBAC: Adding query filter params (role.id=656b8aa0fc5e4f97b03a4251472f521b, user.id=af049965a3514d7dac8cbdbc69963392, scope.system=all) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:05.730816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:05.732972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:05.737263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:05.741324 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 144/282] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 30 09:22:05 2022] GET /identity/v3/role_assignments?user.id=af049965a3514d7dac8cbdbc69963392&scope.system=all&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:05.831914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aeaaec-add0-4a6b-b5ed-fe256d89f025 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:05.873035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:05.873685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:05.873970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:05.874241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:05.874926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:05.875301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:05.876765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:05.887081 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 139/283] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:05.893478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:05.963093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:06.048759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e95e8-d869-4eb4-8a44-f56e4816d70e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:06.099200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643f1d3-6065-4dae-b07d-f9746f791a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:06.099851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643f1d3-6065-4dae-b07d-f9746f791a29 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:06.100133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643f1d3-6065-4dae-b07d-f9746f791a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:06.100411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643f1d3-6065-4dae-b07d-f9746f791a29 None admin] PATH_INFO: `/v3/users/7b3945bdedbc4362a2a9a1f55599a164` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:06.101598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643f1d3-6065-4dae-b07d-f9746f791a29 None admin] RBAC: Authorizing `identity:get_user(user_id=7b3945bdedbc4362a2a9a1f55599a164)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:06.103099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643f1d3-6065-4dae-b07d-f9746f791a29 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:06.104709 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 145/284] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:05 2022] GET /identity/v3/users/7b3945bdedbc4362a2a9a1f55599a164 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:06.110323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:06.153471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632504e-47f2-405a-8d4b-423638072442 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:06.153814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632504e-47f2-405a-8d4b-423638072442 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:06.154057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632504e-47f2-405a-8d4b-423638072442 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:06.155399 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 146/285] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:06.160860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bc62e-6d07-473c-83da-b341c718ad29 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:06.161139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bc62e-6d07-473c-83da-b341c718ad29 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:06.161616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bc62e-6d07-473c-83da-b341c718ad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:06.193970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-281bc62e-6d07-473c-83da-b341c718ad29 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:06.195160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-281bc62e-6d07-473c-83da-b341c718ad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:06.280315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:06.290711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3627091-35e7-4c05-aecd-296460f7f67c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:06.327080 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 147/286] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:06.349462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc85ee-6a08-48d0-9289-e7909113bb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:06.350620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc85ee-6a08-48d0-9289-e7909113bb74 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:06.351093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc85ee-6a08-48d0-9289-e7909113bb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:06.351585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc85ee-6a08-48d0-9289-e7909113bb74 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:06.357328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc85ee-6a08-48d0-9289-e7909113bb74 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:06.359019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc85ee-6a08-48d0-9289-e7909113bb74 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:06.360869 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 140/287] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:06 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:06.366730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bc62e-6d07-473c-83da-b341c718ad29 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:06.433293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:06.512443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bc62e-6d07-473c-83da-b341c718ad29 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:06.553949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83fb2d-115d-4c48-9325-031e591dce21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:06.554615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83fb2d-115d-4c48-9325-031e591dce21 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:06.554891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83fb2d-115d-4c48-9325-031e591dce21 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:06.555156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83fb2d-115d-4c48-9325-031e591dce21 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/users/7b3945bdedbc4362a2a9a1f55599a164/roles/2efac5a86ee940c49657b4c67e2d12e6` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:06.557079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83fb2d-115d-4c48-9325-031e591dce21 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user_id=7b3945bdedbc4362a2a9a1f55599a164, role_id=2efac5a86ee940c49657b4c67e2d12e6)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:06.558570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83fb2d-115d-4c48-9325-031e591dce21 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:06.584065 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 148/288] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:22:06 2022] PUT /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/users/7b3945bdedbc4362a2a9a1f55599a164/roles/2efac5a86ee940c49657b4c67e2d12e6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:22:06.712985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc85ee-6a08-48d0-9289-e7909113bb74 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:06.849854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc85ee-6a08-48d0-9289-e7909113bb74 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:06.894684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c1bc9-2ba5-4c3f-8940-a8fce170bade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:06.895322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1bc9-2ba5-4c3f-8940-a8fce170bade None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:06.895706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1bc9-2ba5-4c3f-8940-a8fce170bade None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:06.895981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c1bc9-2ba5-4c3f-8940-a8fce170bade None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:06.909477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c1bc9-2ba5-4c3f-8940-a8fce170bade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3b8c1bc9-2ba5-4c3f-8940-a8fce170bade None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:06.914826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:06.915677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:06.916461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:06.917519 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 141/289] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:06.922485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83fb2d-115d-4c48-9325-031e591dce21 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:07.061992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83fb2d-115d-4c48-9325-031e591dce21 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:07.071135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:07.106597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:07.107107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:07.107382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:07.107649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:07.108379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:07.108681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:07.110091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:07.119198 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 149/290] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:07.124637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c1bc9-2ba5-4c3f-8940-a8fce170bade None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:07.164078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:07.258123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c1bc9-2ba5-4c3f-8940-a8fce170bade None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:07.293285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da25f2c-7bbd-4bfc-8c73-199aaf38f667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:07.293952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da25f2c-7bbd-4bfc-8c73-199aaf38f667 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:07.294229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da25f2c-7bbd-4bfc-8c73-199aaf38f667 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:07.294502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da25f2c-7bbd-4bfc-8c73-199aaf38f667 None admin] PATH_INFO: `/v3/users/cc1888a6562e46d0a17a64cc3ce55e9d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:07.295630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da25f2c-7bbd-4bfc-8c73-199aaf38f667 None admin] RBAC: Authorizing `identity:get_user(user_id=cc1888a6562e46d0a17a64cc3ce55e9d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:07.297096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da25f2c-7bbd-4bfc-8c73-199aaf38f667 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:07.303434 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 142/291] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:07 2022] GET /identity/v3/users/cc1888a6562e46d0a17a64cc3ce55e9d => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:07.304964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:07.406016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e489672-0ff6-440b-a913-e068d19c7f08 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:07.459271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339df930-0be4-4f59-ad66-6d741664cf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:07.459891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339df930-0be4-4f59-ad66-6d741664cf97 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:07.460166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339df930-0be4-4f59-ad66-6d741664cf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:07.460437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339df930-0be4-4f59-ad66-6d741664cf97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:07.507521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:07.536616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339df930-0be4-4f59-ad66-6d741664cf97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:07.538131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339df930-0be4-4f59-ad66-6d741664cf97 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:07.654969 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 150/292] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:07.660429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da25f2c-7bbd-4bfc-8c73-199aaf38f667 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:07.758369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:07.768332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da25f2c-7bbd-4bfc-8c73-199aaf38f667 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:07.811521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff7ed8-4459-437a-b42e-01f77190c257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:07.812153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff7ed8-4459-437a-b42e-01f77190c257 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:07.812430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff7ed8-4459-437a-b42e-01f77190c257 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:07.812700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff7ed8-4459-437a-b42e-01f77190c257 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:07.813849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff7ed8-4459-437a-b42e-01f77190c257 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:07.815297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff7ed8-4459-437a-b42e-01f77190c257 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:07.821740 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 143/293] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:07 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:07.824127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339df930-0be4-4f59-ad66-6d741664cf97 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:07.929202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:07.970933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339df930-0be4-4f59-ad66-6d741664cf97 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:08.022533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442d3bf-8703-4876-b295-48be027e058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:08.023161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442d3bf-8703-4876-b295-48be027e058a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:08.023442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442d3bf-8703-4876-b295-48be027e058a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:08.023713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442d3bf-8703-4876-b295-48be027e058a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:08.024450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442d3bf-8703-4876-b295-48be027e058a None admin] RBAC: Adding query filter params (role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user.id=cc1888a6562e46d0a17a64cc3ce55e9d) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:08.024752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442d3bf-8703-4876-b295-48be027e058a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:08.026241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442d3bf-8703-4876-b295-48be027e058a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:08.032840 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 151/294] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:07 2022] GET /identity/v3/role_assignments?user.id=cc1888a6562e46d0a17a64cc3ce55e9d&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:08.377098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8dde2-ef4b-443a-9fb8-5db0a83988a1 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:08.377654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8dde2-ef4b-443a-9fb8-5db0a83988a1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:08.377654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8dde2-ef4b-443a-9fb8-5db0a83988a1 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:08.379075 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 144/295] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:08.385217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e7691-47a3-4ab8-b0b4-949ef570bca0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:08.385555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e7691-47a3-4ab8-b0b4-949ef570bca0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:08.385820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e7691-47a3-4ab8-b0b4-949ef570bca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:08.418497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-125e7691-47a3-4ab8-b0b4-949ef570bca0 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:08.419704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-125e7691-47a3-4ab8-b0b4-949ef570bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:08.498711 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 152/296] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:08.514180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b617d2-a15f-45da-b389-f822ecab0253 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:08.514467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b617d2-a15f-45da-b389-f822ecab0253 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:08.514723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b617d2-a15f-45da-b389-f822ecab0253 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:08.515331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854eb61-680c-47a9-987a-12461a4efa69 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:08.515616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854eb61-680c-47a9-987a-12461a4efa69 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:08.515886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854eb61-680c-47a9-987a-12461a4efa69 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:08.516317 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 145/297] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:08.517426 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 153/298] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:22:08.522065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343b96-3041-48c6-b014-59a19fa39e08 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:08.522348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343b96-3041-48c6-b014-59a19fa39e08 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:08.522619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343b96-3041-48c6-b014-59a19fa39e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:08.526813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94f8e9-e12a-44f4-918e-6761f6f90989 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:08.527323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94f8e9-e12a-44f4-918e-6761f6f90989 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:08.527797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94f8e9-e12a-44f4-918e-6761f6f90989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:08.547300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-77343b96-3041-48c6-b014-59a19fa39e08 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:08.548190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6c94f8e9-e12a-44f4-918e-6761f6f90989 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:08.548502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-77343b96-3041-48c6-b014-59a19fa39e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:08.549817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6c94f8e9-e12a-44f4-918e-6761f6f90989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:08.631747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:08.636901 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 146/299] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:08.641934 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 154/300] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:08.800220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77343b96-3041-48c6-b014-59a19fa39e08 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:08.892307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:08.909862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77343b96-3041-48c6-b014-59a19fa39e08 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:08.923613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94f8e9-e12a-44f4-918e-6761f6f90989 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:08.949493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00673-a3cd-4bd0-9190-caaaf62a4b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:08.950162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00673-a3cd-4bd0-9190-caaaf62a4b37 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:08.950444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00673-a3cd-4bd0-9190-caaaf62a4b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:08.950719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00673-a3cd-4bd0-9190-caaaf62a4b37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:08.961682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f00673-a3cd-4bd0-9190-caaaf62a4b37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-79f00673-a3cd-4bd0-9190-caaaf62a4b37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:08.966506 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:08.967347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:08.968118 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 147/301] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:08.968743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00673-a3cd-4bd0-9190-caaaf62a4b37 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:09.034122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94f8e9-e12a-44f4-918e-6761f6f90989 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.069420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed611e-1fb2-4be6-ba1e-7b5ff8e8b0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.070098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed611e-1fb2-4be6-ba1e-7b5ff8e8b0bc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.070382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed611e-1fb2-4be6-ba1e-7b5ff8e8b0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.070658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed611e-1fb2-4be6-ba1e-7b5ff8e8b0bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.079276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00673-a3cd-4bd0-9190-caaaf62a4b37 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.080859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed611e-1fb2-4be6-ba1e-7b5ff8e8b0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0aed611e-1fb2-4be6-ba1e-7b5ff8e8b0bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:09.085997 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:09.086879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:09.087740 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 155/302] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:09.088400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed611e-1fb2-4be6-ba1e-7b5ff8e8b0bc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:09.119088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303582f-9a7a-49a8-baef-a40e7d5690ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.119755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303582f-9a7a-49a8-baef-a40e7d5690ec None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.120031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303582f-9a7a-49a8-baef-a40e7d5690ec None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.120302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303582f-9a7a-49a8-baef-a40e7d5690ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.121414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303582f-9a7a-49a8-baef-a40e7d5690ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:09.122892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303582f-9a7a-49a8-baef-a40e7d5690ec None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:09.125169 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 148/303] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:09.125526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303582f-9a7a-49a8-baef-a40e7d5690ec None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:09.143749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:09.150759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:09.194734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed611e-1fb2-4be6-ba1e-7b5ff8e8b0bc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.228667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.229298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.229543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.229842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.230547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:09.230803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:09.232177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:09.241562 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 156/304] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:09.242207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:09.250061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303582f-9a7a-49a8-baef-a40e7d5690ec None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.289165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.289827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.290096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.290372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.291043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:09.291350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:09.292732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:09.301254 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 149/305] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:09.302155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:09.358786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bfe40-3bd5-4601-bd48-e5c750b12a96 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.377222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:09.389178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:09.399972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e782d-af14-4244-945c-3b010c864558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.400509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e782d-af14-4244-945c-3b010c864558 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.400952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e782d-af14-4244-945c-3b010c864558 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.401059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e782d-af14-4244-945c-3b010c864558 None admin] PATH_INFO: `/v3/groups` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.401962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e782d-af14-4244-945c-3b010c864558 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:09.404693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e782d-af14-4244-945c-3b010c864558 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:09.419622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc478ca-1c6c-471f-8f7b-3d0c4a6b9eee None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.420589 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 157/306] 10.222.0.15 () {60 vars in 1134 bytes} [Fri Sep 30 09:22:09 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:22:09.421237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e782d-af14-4244-945c-3b010c864558 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:09.473117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab1f93-3655-4afe-a9a9-bfc620265ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.474596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab1f93-3655-4afe-a9a9-bfc620265ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.475557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab1f93-3655-4afe-a9a9-bfc620265ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.475841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab1f93-3655-4afe-a9a9-bfc620265ba2 None admin] PATH_INFO: `/v3/users/af049965a3514d7dac8cbdbc69963392` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.477158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab1f93-3655-4afe-a9a9-bfc620265ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=af049965a3514d7dac8cbdbc69963392)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:09.478692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab1f93-3655-4afe-a9a9-bfc620265ba2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:09.480552 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 150/307] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:09 2022] GET /identity/v3/users/af049965a3514d7dac8cbdbc69963392 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:09.486704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d8c66-6dc0-4261-be09-119437f32c0a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.487332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d8c66-6dc0-4261-be09-119437f32c0a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.487332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d8c66-6dc0-4261-be09-119437f32c0a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.488732 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 151/308] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:09.489322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d8c66-6dc0-4261-be09-119437f32c0a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:09.546956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e782d-af14-4244-945c-3b010c864558 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.585483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d773e-fa0b-4e2f-b2b4-7f1e16fa1e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.586160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d773e-fa0b-4e2f-b2b4-7f1e16fa1e4c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.586447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d773e-fa0b-4e2f-b2b4-7f1e16fa1e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.586719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d773e-fa0b-4e2f-b2b4-7f1e16fa1e4c None admin] PATH_INFO: `/v3/users/267ed5b5eac14f1cab2f370c6aca5b10` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.601365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d773e-fa0b-4e2f-b2b4-7f1e16fa1e4c None admin] RBAC: Authorizing `identity:get_user(user_id=267ed5b5eac14f1cab2f370c6aca5b10)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:09.603233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d773e-fa0b-4e2f-b2b4-7f1e16fa1e4c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:09.604724 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 158/309] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:09 2022] GET /identity/v3/users/267ed5b5eac14f1cab2f370c6aca5b10 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:09.605716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8db32-f8fa-42db-aded-fc2e88043dae None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.606002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8db32-f8fa-42db-aded-fc2e88043dae None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.606274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8db32-f8fa-42db-aded-fc2e88043dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.616094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d8c66-6dc0-4261-be09-119437f32c0a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.627083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9ac8db32-f8fa-42db-aded-fc2e88043dae None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:09.628569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ac8db32-f8fa-42db-aded-fc2e88043dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:09.651262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebd301-0f19-40c8-8bb0-9aea546c81e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.652106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebd301-0f19-40c8-8bb0-9aea546c81e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.652383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebd301-0f19-40c8-8bb0-9aea546c81e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.652655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebd301-0f19-40c8-8bb0-9aea546c81e3 None admin] PATH_INFO: `/v3/system/users/af049965a3514d7dac8cbdbc69963392/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.654081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebd301-0f19-40c8-8bb0-9aea546c81e3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=af049965a3514d7dac8cbdbc69963392, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:09.655459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebd301-0f19-40c8-8bb0-9aea546c81e3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:09.659433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:09.667280 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 152/310] 10.222.0.15 () {58 vars in 1334 bytes} [Fri Sep 30 09:22:09 2022] PUT /identity/v3/system/users/af049965a3514d7dac8cbdbc69963392/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:09.668074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebd301-0f19-40c8-8bb0-9aea546c81e3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:09.689730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:09.718702 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 159/311] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:09.805204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebd301-0f19-40c8-8bb0-9aea546c81e3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:09.844193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:09.844810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:09.845089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:09.845361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:09.846115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] RBAC: Adding query filter params (role.id=2efac5a86ee940c49657b4c67e2d12e6, user.id=267ed5b5eac14f1cab2f370c6aca5b10, scope.system=all) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:09.846419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:09.849578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:09.854000 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 153/312] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 30 09:22:09 2022] GET /identity/v3/role_assignments?user.id=267ed5b5eac14f1cab2f370c6aca5b10&scope.system=all&role.id=2efac5a86ee940c49657b4c67e2d12e6 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:10.013432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8db32-f8fa-42db-aded-fc2e88043dae None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:10.124680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8db32-f8fa-42db-aded-fc2e88043dae None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:10.162004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fb413-c246-423a-a179-3c63008da24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:10.162646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fb413-c246-423a-a179-3c63008da24e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:10.162930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fb413-c246-423a-a179-3c63008da24e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:10.163204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fb413-c246-423a-a179-3c63008da24e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:10.174049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fb413-c246-423a-a179-3c63008da24e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-679fb413-c246-423a-a179-3c63008da24e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:10.179140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:10.180005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:10.180773 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 160/313] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:10.186227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:10.265799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:10.291212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ef7b8-2949-4671-a63d-06bfde032ab9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:10.325574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:10.326209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:10.326481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:10.326749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:10.327410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:10.327707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:10.329075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:10.337700 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 154/314] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:10.343540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fb413-c246-423a-a179-3c63008da24e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:10.378691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:10.457376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fb413-c246-423a-a179-3c63008da24e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:10.493325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c4b16-a65b-4f5c-825e-61d4a47bf0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:10.494259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c4b16-a65b-4f5c-825e-61d4a47bf0ac None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:10.494681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c4b16-a65b-4f5c-825e-61d4a47bf0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:10.495117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c4b16-a65b-4f5c-825e-61d4a47bf0ac None admin] PATH_INFO: `/v3/users/7b3945bdedbc4362a2a9a1f55599a164` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:10.496468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c4b16-a65b-4f5c-825e-61d4a47bf0ac None admin] RBAC: Authorizing `identity:get_user(user_id=7b3945bdedbc4362a2a9a1f55599a164)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:10.498079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c4b16-a65b-4f5c-825e-61d4a47bf0ac None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:10.503704 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 161/315] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:10 2022] GET /identity/v3/users/7b3945bdedbc4362a2a9a1f55599a164 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:10.506151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:10.608503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9307c0-2de9-4b2f-b214-35c3abd05a47 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:10.644777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7992ade-b1fe-4dff-aa5c-02865ea0572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:10.645385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7992ade-b1fe-4dff-aa5c-02865ea0572e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:10.646304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7992ade-b1fe-4dff-aa5c-02865ea0572e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:10.646576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7992ade-b1fe-4dff-aa5c-02865ea0572e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:10.650088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:10.714889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7992ade-b1fe-4dff-aa5c-02865ea0572e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:10.716355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7992ade-b1fe-4dff-aa5c-02865ea0572e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:10.818758 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 155/316] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:10.824577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c4b16-a65b-4f5c-825e-61d4a47bf0ac None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:10.917838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:10.926395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c4b16-a65b-4f5c-825e-61d4a47bf0ac None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:10.960934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db7d70-75de-468b-868e-14efde27027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:10.961561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7d70-75de-468b-868e-14efde27027f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:10.961868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7d70-75de-468b-868e-14efde27027f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:10.962141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db7d70-75de-468b-868e-14efde27027f None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:10.963260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db7d70-75de-468b-868e-14efde27027f None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:10.964707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db7d70-75de-468b-868e-14efde27027f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:10.966349 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 162/317] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:10 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:10.972087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7992ade-b1fe-4dff-aa5c-02865ea0572e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:11.007706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:11.073733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7992ade-b1fe-4dff-aa5c-02865ea0572e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:11.108065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871b040-24fa-4edb-9b17-c0f8060474c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:11.108674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871b040-24fa-4edb-9b17-c0f8060474c6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:11.108949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871b040-24fa-4edb-9b17-c0f8060474c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:11.109215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871b040-24fa-4edb-9b17-c0f8060474c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:11.109962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871b040-24fa-4edb-9b17-c0f8060474c6 None admin] RBAC: Adding query filter params (role.id=2efac5a86ee940c49657b4c67e2d12e6, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e, user.id=7b3945bdedbc4362a2a9a1f55599a164) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:11.110267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871b040-24fa-4edb-9b17-c0f8060474c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:11.111706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871b040-24fa-4edb-9b17-c0f8060474c6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:11.118093 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 156/318] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:10 2022] GET /identity/v3/role_assignments?user.id=7b3945bdedbc4362a2a9a1f55599a164&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=2efac5a86ee940c49657b4c67e2d12e6 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:11.733527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089fd4-40d3-4efa-a7c0-e885e1d91f5c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:11.734076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089fd4-40d3-4efa-a7c0-e885e1d91f5c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:11.734076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089fd4-40d3-4efa-a7c0-e885e1d91f5c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:11.735385 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 163/319] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:11.740713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5028c-4e2c-4106-9958-6547530b0373 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:11.741114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5028c-4e2c-4106-9958-6547530b0373 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:11.741300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5028c-4e2c-4106-9958-6547530b0373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:11.759707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-87a5028c-4e2c-4106-9958-6547530b0373 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:11.760875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-87a5028c-4e2c-4106-9958-6547530b0373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:11.836921 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 157/320] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:11.911589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1dfca-0acf-4be0-9c0a-116e54315ce4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:11.911889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1dfca-0acf-4be0-9c0a-116e54315ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:11.912141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1dfca-0acf-4be0-9c0a-116e54315ce4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:11.913617 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 164/321] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:11.918680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3024a9-5fcc-4310-99bd-898039d1ddc3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:11.918971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3024a9-5fcc-4310-99bd-898039d1ddc3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:11.919250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3024a9-5fcc-4310-99bd-898039d1ddc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:11.937994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fd3024a9-5fcc-4310-99bd-898039d1ddc3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:11.939184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fd3024a9-5fcc-4310-99bd-898039d1ddc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:11.964023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:12.013747 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 158/322] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:12.108741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1dfca-0acf-4be0-9c0a-116e54315ce4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:12.157024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6acf47-a0a8-4e87-8a32-98cb9ffa529c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.157362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6acf47-a0a8-4e87-8a32-98cb9ffa529c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.157584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6acf47-a0a8-4e87-8a32-98cb9ffa529c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.159089 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 159/323] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:12.163964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5e38-345d-4e08-a234-98b83de101b9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.164248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5e38-345d-4e08-a234-98b83de101b9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.164520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5e38-345d-4e08-a234-98b83de101b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.184480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7d4a5e38-345d-4e08-a234-98b83de101b9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:12.185689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7d4a5e38-345d-4e08-a234-98b83de101b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:12.215281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1dfca-0acf-4be0-9c0a-116e54315ce4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:12.251481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e7eaf-e82b-4c6f-a2de-f687dc3d9344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:12.252390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e7eaf-e82b-4c6f-a2de-f687dc3d9344 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.252669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e7eaf-e82b-4c6f-a2de-f687dc3d9344 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.252938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e7eaf-e82b-4c6f-a2de-f687dc3d9344 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.254153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e7eaf-e82b-4c6f-a2de-f687dc3d9344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:12.256265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e7eaf-e82b-4c6f-a2de-f687dc3d9344 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:12.258007 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 165/324] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:12.263390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e7eaf-e82b-4c6f-a2de-f687dc3d9344 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:12.265940 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 160/325] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:12.283409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a5e38-345d-4e08-a234-98b83de101b9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:12.293810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:12.372567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e7eaf-e82b-4c6f-a2de-f687dc3d9344 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:12.376482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:12.387294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a5e38-345d-4e08-a234-98b83de101b9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:12.413910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603437c6-eef8-488b-8377-ebc836cb6c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:12.414623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603437c6-eef8-488b-8377-ebc836cb6c10 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.414833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603437c6-eef8-488b-8377-ebc836cb6c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.415055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603437c6-eef8-488b-8377-ebc836cb6c10 None admin] PATH_INFO: `/v3/groups` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.415686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603437c6-eef8-488b-8377-ebc836cb6c10 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:12.417141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603437c6-eef8-488b-8377-ebc836cb6c10 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:12.434608 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 166/326] 10.222.0.15 () {60 vars in 1134 bytes} [Fri Sep 30 09:22:12 2022] POST /identity/v3/groups => generated 226 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:22:12.449444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7151ea8-d34d-42ef-9158-8a83f2fe7ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:12.450275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7151ea8-d34d-42ef-9158-8a83f2fe7ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.450614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7151ea8-d34d-42ef-9158-8a83f2fe7ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.450949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7151ea8-d34d-42ef-9158-8a83f2fe7ef0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.463815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7151ea8-d34d-42ef-9158-8a83f2fe7ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-c7151ea8-d34d-42ef-9158-8a83f2fe7ef0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:12.470027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:12.470973 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:12.474660 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 161/327] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:12.477990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603437c6-eef8-488b-8377-ebc836cb6c10 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:12.543708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7151ea8-d34d-42ef-9158-8a83f2fe7ef0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:12.585611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:12.601127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603437c6-eef8-488b-8377-ebc836cb6c10 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:12.643807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:12.643807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.643807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.644145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.645167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:12.645667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:12.648164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:12.659236 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 167/328] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:12.661711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7151ea8-d34d-42ef-9158-8a83f2fe7ef0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:12.666961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:12.702279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25a86b-b2ed-4ec3-b0b4-3d1238988a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:12.703413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a86b-b2ed-4ec3-b0b4-3d1238988a70 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.703413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a86b-b2ed-4ec3-b0b4-3d1238988a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.703630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a86b-b2ed-4ec3-b0b4-3d1238988a70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.720725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25a86b-b2ed-4ec3-b0b4-3d1238988a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ec25a86b-b2ed-4ec3-b0b4-3d1238988a70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:12.729476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:12.730623 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:12.731854 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 162/329] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:12.736586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25a86b-b2ed-4ec3-b0b4-3d1238988a70 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:12.751838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:12.804328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ef97b-7e4a-47c6-bce6-2665c5b11186 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:12.847704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b850fc0-84c9-4d5b-99cc-cc274beb2d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:12.848470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b850fc0-84c9-4d5b-99cc-cc274beb2d01 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.849320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b850fc0-84c9-4d5b-99cc-cc274beb2d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.849595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b850fc0-84c9-4d5b-99cc-cc274beb2d01 None admin] PATH_INFO: `/v3/users/af049965a3514d7dac8cbdbc69963392` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.851909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b850fc0-84c9-4d5b-99cc-cc274beb2d01 None admin] RBAC: Authorizing `identity:get_user(user_id=af049965a3514d7dac8cbdbc69963392)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:12.853490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b850fc0-84c9-4d5b-99cc-cc274beb2d01 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:12.855935 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 168/330] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:12 2022] GET /identity/v3/users/af049965a3514d7dac8cbdbc69963392 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:12.861954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b850fc0-84c9-4d5b-99cc-cc274beb2d01 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:12.874778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25a86b-b2ed-4ec3-b0b4-3d1238988a70 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:12.890513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:12.920976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:12.921598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:12.921950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:12.922221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:12.923058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:12.923357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:12.924731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:12.936380 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 163/331] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:12.941707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:13.007578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b850fc0-84c9-4d5b-99cc-cc274beb2d01 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:13.043353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:13.059904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:13.060704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:13.060991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:13.061264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:13.062029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] RBAC: Adding query filter params (role.id=656b8aa0fc5e4f97b03a4251472f521b, user.id=af049965a3514d7dac8cbdbc69963392, scope.system=all) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:13.062337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:13.064050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:13.078486 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 169/332] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 30 09:22:12 2022] GET /identity/v3/role_assignments?user.id=af049965a3514d7dac8cbdbc69963392&scope.system=all&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:13.102298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce1f55-28f4-4c3b-ba5f-131cf646b4d0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:13.141772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fecad4-6421-45d4-8ec7-13ed2c4fcd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:13.142425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fecad4-6421-45d4-8ec7-13ed2c4fcd26 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:13.142705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fecad4-6421-45d4-8ec7-13ed2c4fcd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:13.142972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fecad4-6421-45d4-8ec7-13ed2c4fcd26 None admin] PATH_INFO: `/v3/users/267ed5b5eac14f1cab2f370c6aca5b10` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:13.144125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fecad4-6421-45d4-8ec7-13ed2c4fcd26 None admin] RBAC: Authorizing `identity:get_user(user_id=267ed5b5eac14f1cab2f370c6aca5b10)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:13.146241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fecad4-6421-45d4-8ec7-13ed2c4fcd26 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:13.148315 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 164/333] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:12 2022] GET /identity/v3/users/267ed5b5eac14f1cab2f370c6aca5b10 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:13.154241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:13.271511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:13.298574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e21a8-8259-4aaa-a0ea-d1f14cd3fc7a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:13.341850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3575a-29de-489e-aecd-7f44318b0049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:13.342501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3575a-29de-489e-aecd-7f44318b0049 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:13.342788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3575a-29de-489e-aecd-7f44318b0049 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:13.343066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3575a-29de-489e-aecd-7f44318b0049 None admin] PATH_INFO: `/v3/system/users/267ed5b5eac14f1cab2f370c6aca5b10/roles/2efac5a86ee940c49657b4c67e2d12e6` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:13.344556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3575a-29de-489e-aecd-7f44318b0049 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=267ed5b5eac14f1cab2f370c6aca5b10, role_id=2efac5a86ee940c49657b4c67e2d12e6)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:13.346081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3575a-29de-489e-aecd-7f44318b0049 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:13.359027 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 170/334] 10.222.0.15 () {58 vars in 1334 bytes} [Fri Sep 30 09:22:13 2022] PUT /identity/v3/system/users/267ed5b5eac14f1cab2f370c6aca5b10/roles/2efac5a86ee940c49657b4c67e2d12e6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:15.121735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb868-ada6-4c66-865e-20e3d5c30f4b None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:15.122765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb868-ada6-4c66-865e-20e3d5c30f4b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:15.123046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb868-ada6-4c66-865e-20e3d5c30f4b None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:15.124550 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 165/335] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:15.130581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29b980-b87d-46d6-9153-e689ac73b34f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:15.131547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29b980-b87d-46d6-9153-e689ac73b34f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:15.132026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29b980-b87d-46d6-9153-e689ac73b34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:15.151814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-df29b980-b87d-46d6-9153-e689ac73b34f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:15.153451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-df29b980-b87d-46d6-9153-e689ac73b34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:15.232684 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 171/336] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:15.532616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb868-ada6-4c66-865e-20e3d5c30f4b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:15.662079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb868-ada6-4c66-865e-20e3d5c30f4b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:15.711678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e48fa-34c1-47df-ad13-4382a5a6c9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:15.712692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e48fa-34c1-47df-ad13-4382a5a6c9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:15.713272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e48fa-34c1-47df-ad13-4382a5a6c9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:15.713781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e48fa-34c1-47df-ad13-4382a5a6c9e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:15.724444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:15.728699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e48fa-34c1-47df-ad13-4382a5a6c9e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-be2e48fa-34c1-47df-ad13-4382a5a6c9e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:15.733717 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:15.734667 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:15.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 166/337] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:22:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:15.741829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29b980-b87d-46d6-9153-e689ac73b34f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:15.786812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef6565-bd75-4fda-885d-5ce36412426d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:15.787088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef6565-bd75-4fda-885d-5ce36412426d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:15.787461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef6565-bd75-4fda-885d-5ce36412426d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:15.788990 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 167/338] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:15.802770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0642a-88c9-4edc-9122-5ff5f55ad3fc None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:15.803072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0642a-88c9-4edc-9122-5ff5f55ad3fc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:15.803323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0642a-88c9-4edc-9122-5ff5f55ad3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:15.837957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-70c0642a-88c9-4edc-9122-5ff5f55ad3fc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:15.839153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-70c0642a-88c9-4edc-9122-5ff5f55ad3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:15.864548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:15.928614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29b980-b87d-46d6-9153-e689ac73b34f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:15.971886 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 168/339] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:15.995635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:15.996255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:15.996598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:15.996861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:15.997531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:15.997870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:15.999347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:16.008618 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 172/340] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:16.014256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0642a-88c9-4edc-9122-5ff5f55ad3fc None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:16.130900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0642a-88c9-4edc-9122-5ff5f55ad3fc None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:16.166699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:16.173873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d25e2-1041-45c8-bb35-63acac8b02b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:16.174590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d25e2-1041-45c8-bb35-63acac8b02b4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.174938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d25e2-1041-45c8-bb35-63acac8b02b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.175216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d25e2-1041-45c8-bb35-63acac8b02b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.264305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d25e2-1041-45c8-bb35-63acac8b02b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:16.265850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d25e2-1041-45c8-bb35-63acac8b02b4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:16.382912 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 169/341] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:16.388204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:16.415879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf490a2-43ed-4121-9ef4-e440a6696b30 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.416617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf490a2-43ed-4121-9ef4-e440a6696b30 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.417092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf490a2-43ed-4121-9ef4-e440a6696b30 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.418649 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 170/342] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:16.435525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4455ec1-d244-4557-8864-39c1c3ab3a17 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.436139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4455ec1-d244-4557-8864-39c1c3ab3a17 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.436633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4455ec1-d244-4557-8864-39c1c3ab3a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.467291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e4455ec1-d244-4557-8864-39c1c3ab3a17 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:16.468854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e4455ec1-d244-4557-8864-39c1c3ab3a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:16.507009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:16.555312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b0678-9bc4-4886-a4a5-f883c67777dd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:16.571623 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 171/343] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:16.573219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4720a8-ea11-4d9b-9658-97d35cbbe429 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.573695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4720a8-ea11-4d9b-9658-97d35cbbe429 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.574184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4720a8-ea11-4d9b-9658-97d35cbbe429 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.576439 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 172/344] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:16.577281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4720a8-ea11-4d9b-9658-97d35cbbe429 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:16.605148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:16.611277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ac0a2-061e-4ee3-9fde-b5b3f1545fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:16.613946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ac0a2-061e-4ee3-9fde-b5b3f1545fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.614594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ac0a2-061e-4ee3-9fde-b5b3f1545fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.615017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ac0a2-061e-4ee3-9fde-b5b3f1545fe8 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.616400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ac0a2-061e-4ee3-9fde-b5b3f1545fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:16.618044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ac0a2-061e-4ee3-9fde-b5b3f1545fe8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:16.619929 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 173/345] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:16 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:16.621256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36de3a-7c6f-41a7-aec6-99f20777fef1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.621800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36de3a-7c6f-41a7-aec6-99f20777fef1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.622286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36de3a-7c6f-41a7-aec6-99f20777fef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.650880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6d36de3a-7c6f-41a7-aec6-99f20777fef1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:16.652683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6d36de3a-7c6f-41a7-aec6-99f20777fef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:16.691965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4720a8-ea11-4d9b-9658-97d35cbbe429 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:16.728565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114640a6-d798-4bcd-84d3-4ed79f754589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:16.729205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114640a6-d798-4bcd-84d3-4ed79f754589 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.729485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114640a6-d798-4bcd-84d3-4ed79f754589 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.729785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114640a6-d798-4bcd-84d3-4ed79f754589 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.731565 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 174/346] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:16.732603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36de3a-7c6f-41a7-aec6-99f20777fef1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:16.745072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114640a6-d798-4bcd-84d3-4ed79f754589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-114640a6-d798-4bcd-84d3-4ed79f754589 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:16.750109 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:16.750977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:16.751765 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 173/347] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:16.757839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114640a6-d798-4bcd-84d3-4ed79f754589 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:16.781344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:16.845620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36de3a-7c6f-41a7-aec6-99f20777fef1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:16.868094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114640a6-d798-4bcd-84d3-4ed79f754589 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:16.878449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:16.886961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bcd38-42c2-43c0-b8c3-8955650ffa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:16.887875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bcd38-42c2-43c0-b8c3-8955650ffa6e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.888376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bcd38-42c2-43c0-b8c3-8955650ffa6e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.888863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bcd38-42c2-43c0-b8c3-8955650ffa6e None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.895366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bcd38-42c2-43c0-b8c3-8955650ffa6e None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:16.897410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bcd38-42c2-43c0-b8c3-8955650ffa6e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:16.899288 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 175/348] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:16 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:16.905542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bcd38-42c2-43c0-b8c3-8955650ffa6e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:16.914956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:16.915578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:16.915849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:16.916172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:16.916838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:16.917227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:16.920127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:16.935832 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 174/349] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:16.940250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:17.103069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:17.108635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bcd38-42c2-43c0-b8c3-8955650ffa6e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:17.138158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a72288-d578-4dac-b306-dc8684e430e3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:17.166909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:17.167529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:17.167813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:17.168639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:17.169488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] RBAC: Adding query filter params (group.id=4e5ddb7294b24d44a68fd2e6de00c49c, role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:17.169836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:17.171420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:17.178504 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 176/350] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:16 2022] GET /identity/v3/role_assignments?group.id=4e5ddb7294b24d44a68fd2e6de00c49c&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:17.179700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:17.192169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0dcec-c66f-44d6-a708-78d1cc1ccda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:17.192813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0dcec-c66f-44d6-a708-78d1cc1ccda8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:17.193086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0dcec-c66f-44d6-a708-78d1cc1ccda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:17.193351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0dcec-c66f-44d6-a708-78d1cc1ccda8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:17.266001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:17.316368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ece2b-92ec-4874-8a76-7635e81a282f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:17.323067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0dcec-c66f-44d6-a708-78d1cc1ccda8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:17.324569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0dcec-c66f-44d6-a708-78d1cc1ccda8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:17.365925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958b257-626f-403e-b60e-6b2add774176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:17.366476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b257-626f-403e-b60e-6b2add774176 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:17.366743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b257-626f-403e-b60e-6b2add774176 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:17.367017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b257-626f-403e-b60e-6b2add774176 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:17.381522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958b257-626f-403e-b60e-6b2add774176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e958b257-626f-403e-b60e-6b2add774176 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:17.388223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:17.389865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:17.390661 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 177/351] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:22:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:17.391863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958b257-626f-403e-b60e-6b2add774176 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:17.446384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:17.482595 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 175/352] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 09:22:17.484066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0dcec-c66f-44d6-a708-78d1cc1ccda8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:17.565407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958b257-626f-403e-b60e-6b2add774176 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:17.606086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:17.632488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e83eb-7550-4fc4-a8f8-57f4f54d0f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:17.633408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e83eb-7550-4fc4-a8f8-57f4f54d0f8f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:17.633934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e83eb-7550-4fc4-a8f8-57f4f54d0f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:17.634276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0dcec-c66f-44d6-a708-78d1cc1ccda8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:17.634513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e83eb-7550-4fc4-a8f8-57f4f54d0f8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:17.656306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e83eb-7550-4fc4-a8f8-57f4f54d0f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6e1e83eb-7550-4fc4-a8f8-57f4f54d0f8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:17.665065 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:17.666226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:17.667015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:17.669284 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 178/353] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:17.670778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8a235-3017-41f8-b979-dda7a19c9e63 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:17.671237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8a235-3017-41f8-b979-dda7a19c9e63 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:17.671693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8a235-3017-41f8-b979-dda7a19c9e63 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:17.673407 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 179/354] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:17.674508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8a235-3017-41f8-b979-dda7a19c9e63 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:17.684070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eec3b-2836-4f14-8a7b-7c4bb9a2184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:17.684813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eec3b-2836-4f14-8a7b-7c4bb9a2184d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:17.685159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eec3b-2836-4f14-8a7b-7c4bb9a2184d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:17.685505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eec3b-2836-4f14-8a7b-7c4bb9a2184d None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:17.686279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1eec3b-2836-4f14-8a7b-7c4bb9a2184d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:17.686653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1eec3b-2836-4f14-8a7b-7c4bb9a2184d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:17.689010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1eec3b-2836-4f14-8a7b-7c4bb9a2184d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:17.706671 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 176/355] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:17.709239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213502e0-5bd9-4dea-8db7-9161ad18895f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:17.709595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213502e0-5bd9-4dea-8db7-9161ad18895f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:17.709984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213502e0-5bd9-4dea-8db7-9161ad18895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:17.733120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-213502e0-5bd9-4dea-8db7-9161ad18895f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:17.734983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-213502e0-5bd9-4dea-8db7-9161ad18895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:17.806789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:17.840446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8a235-3017-41f8-b979-dda7a19c9e63 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:17.851907 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 177/356] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:17.853160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213502e0-5bd9-4dea-8db7-9161ad18895f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:17.890802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abba3b-3543-4f28-a746-69e4f8c825e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:17.891351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abba3b-3543-4f28-a746-69e4f8c825e5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:17.891659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abba3b-3543-4f28-a746-69e4f8c825e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:17.892088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abba3b-3543-4f28-a746-69e4f8c825e5 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:17.894077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abba3b-3543-4f28-a746-69e4f8c825e5 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:17.896836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abba3b-3543-4f28-a746-69e4f8c825e5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:17.899203 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 180/357] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:17 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:17.899995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abba3b-3543-4f28-a746-69e4f8c825e5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:17.964863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:17.992882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213502e0-5bd9-4dea-8db7-9161ad18895f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:18.035533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abba3b-3543-4f28-a746-69e4f8c825e5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:18.036135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:18.037351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.038088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.038575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.039510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:18.040117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:18.042419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:18.054610 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 178/358] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:18.055800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:18.089266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054511a-e796-42ac-a551-f83055fc5711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:18.089935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054511a-e796-42ac-a551-f83055fc5711 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.090219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054511a-e796-42ac-a551-f83055fc5711 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.090492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054511a-e796-42ac-a551-f83055fc5711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.111564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:18.187007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054511a-e796-42ac-a551-f83055fc5711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:18.189374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054511a-e796-42ac-a551-f83055fc5711 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:18.193916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8b5b3-12b6-4f8b-901e-0e65f163cc2e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:18.236585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3797edf-4a46-4ee0-b05b-839d4da441c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:18.237691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797edf-4a46-4ee0-b05b-839d4da441c0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.238180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797edf-4a46-4ee0-b05b-839d4da441c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.238672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3797edf-4a46-4ee0-b05b-839d4da441c0 None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.240958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3797edf-4a46-4ee0-b05b-839d4da441c0 None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:18.244373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3797edf-4a46-4ee0-b05b-839d4da441c0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:18.246911 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 179/359] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:18 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:18.247915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3797edf-4a46-4ee0-b05b-839d4da441c0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:18.252948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:18.321640 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 181/360] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:18.323866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea343d1-8da5-4f92-b496-b6ecd14cddb1 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.324413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea343d1-8da5-4f92-b496-b6ecd14cddb1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.324907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea343d1-8da5-4f92-b496-b6ecd14cddb1 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.327483 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 182/361] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:18.328831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea343d1-8da5-4f92-b496-b6ecd14cddb1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:18.386374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3797edf-4a46-4ee0-b05b-839d4da441c0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:18.389881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:18.435657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc49aa-48f0-48d4-b16b-3636a3e4bef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:18.435884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc49aa-48f0-48d4-b16b-3636a3e4bef3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.436121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc49aa-48f0-48d4-b16b-3636a3e4bef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.436383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc49aa-48f0-48d4-b16b-3636a3e4bef3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.463200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea343d1-8da5-4f92-b496-b6ecd14cddb1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:18.508891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:18.509520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.509865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.510139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.510928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] RBAC: Adding query filter params (group.id=4e5ddb7294b24d44a68fd2e6de00c49c, role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:18.511267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:18.512773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:18.524012 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 183/362] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:18 2022] GET /identity/v3/role_assignments?group.id=4e5ddb7294b24d44a68fd2e6de00c49c&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:18.525256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:18.532938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc49aa-48f0-48d4-b16b-3636a3e4bef3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:18.537060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc49aa-48f0-48d4-b16b-3636a3e4bef3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:18.559489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:18.662969 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 180/363] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:18.664509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0507-0726-4168-a65f-3cf22f3925c1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.664792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0507-0726-4168-a65f-3cf22f3925c1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.665059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b0507-0726-4168-a65f-3cf22f3925c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.673933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:18.687707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3d9b0507-0726-4168-a65f-3cf22f3925c1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:18.688929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3d9b0507-0726-4168-a65f-3cf22f3925c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:18.698639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242feb73-24ca-4cc7-9c2d-c790caacc11f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:18.750235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ec0cd-fd57-47f8-a0e5-7041422f9ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:18.751471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ec0cd-fd57-47f8-a0e5-7041422f9ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.751954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ec0cd-fd57-47f8-a0e5-7041422f9ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.752617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ec0cd-fd57-47f8-a0e5-7041422f9ef3 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.755106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ec0cd-fd57-47f8-a0e5-7041422f9ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:18.757081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ec0cd-fd57-47f8-a0e5-7041422f9ef3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:18.760025 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 184/364] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:18 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:18.760992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ec0cd-fd57-47f8-a0e5-7041422f9ef3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:18.777125 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 181/365] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:18.778900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b0507-0726-4168-a65f-3cf22f3925c1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:18.855094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:18.886598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ec0cd-fd57-47f8-a0e5-7041422f9ef3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:18.902059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b0507-0726-4168-a65f-3cf22f3925c1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:18.927011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf3342-632e-4745-b1f7-1879703ba777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:18.927656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf3342-632e-4745-b1f7-1879703ba777 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.927932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf3342-632e-4745-b1f7-1879703ba777 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.928200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf3342-632e-4745-b1f7-1879703ba777 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.940847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d447d-b018-48a1-99e0-bcae9aded8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:18.941171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf3342-632e-4745-b1f7-1879703ba777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:18.942117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d447d-b018-48a1-99e0-bcae9aded8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:18.942791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d447d-b018-48a1-99e0-bcae9aded8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:18.943266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d447d-b018-48a1-99e0-bcae9aded8b5 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:18.944694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d447d-b018-48a1-99e0-bcae9aded8b5 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:18.946477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d447d-b018-48a1-99e0-bcae9aded8b5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6baf3342-632e-4745-b1f7-1879703ba777 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:18.947094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:18.947964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 30 09:22:18.948750 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:18.951741 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 182/367] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:18 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:18.951958 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 185/367] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:18.952927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d447d-b018-48a1-99e0-bcae9aded8b5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:18.955925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:18.961999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf3342-632e-4745-b1f7-1879703ba777 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.089924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf3342-632e-4745-b1f7-1879703ba777 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:19.097841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d447d-b018-48a1-99e0-bcae9aded8b5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:19.131558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830f66f-2f7b-4ece-86f4-2b2dae358b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:19.132202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830f66f-2f7b-4ece-86f4-2b2dae358b9e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:19.132478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830f66f-2f7b-4ece-86f4-2b2dae358b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:19.132746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830f66f-2f7b-4ece-86f4-2b2dae358b9e None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:19.133903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830f66f-2f7b-4ece-86f4-2b2dae358b9e None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:19.135379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830f66f-2f7b-4ece-86f4-2b2dae358b9e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:19.141903 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 186/368] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:18 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:19.142845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830f66f-2f7b-4ece-86f4-2b2dae358b9e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.152242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e301a-2045-4cd0-a46a-1fb99ea67aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:19.152895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e301a-2045-4cd0-a46a-1fb99ea67aac None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:19.153176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e301a-2045-4cd0-a46a-1fb99ea67aac None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:19.153451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e301a-2045-4cd0-a46a-1fb99ea67aac None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:19.153621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:19.158448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e301a-2045-4cd0-a46a-1fb99ea67aac None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:19.159911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e301a-2045-4cd0-a46a-1fb99ea67aac None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:19.161828 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 183/369] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:18 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:19.162598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e301a-2045-4cd0-a46a-1fb99ea67aac None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.277763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830f66f-2f7b-4ece-86f4-2b2dae358b9e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:19.300352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:19.330099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:19.330808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:19.331091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:19.331361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:19.332034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:19.332426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:19.334243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:19.339452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e301a-2045-4cd0-a46a-1fb99ea67aac None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:19.345138 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 187/370] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:19.345966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.374965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:19.375583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:19.375867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:19.376140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:19.376868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] RBAC: Adding query filter params (group.id=4e5ddb7294b24d44a68fd2e6de00c49c, role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:19.377174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:19.378700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:19.385047 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 184/371] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:19 2022] GET /identity/v3/role_assignments?group.id=4e5ddb7294b24d44a68fd2e6de00c49c&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:19.385879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.496549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9c708-86e8-43e4-b99e-bede777a46d2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:19.519035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:19.535164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b17cc9-07f3-4361-b360-ededfd15529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:19.535740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b17cc9-07f3-4361-b360-ededfd15529c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:19.536063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b17cc9-07f3-4361-b360-ededfd15529c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:19.536324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b17cc9-07f3-4361-b360-ededfd15529c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:19.538676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:19.554713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b17cc9-07f3-4361-b360-ededfd15529c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-10b17cc9-07f3-4361-b360-ededfd15529c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:19.561448 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:19.562692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 185/372] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:22:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:19.563674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b17cc9-07f3-4361-b360-ededfd15529c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.580017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a98245-0fb5-4489-b809-3202f573416e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:19.627102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:19.628798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:19.629373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:19.630236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:19.631739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] RBAC: Adding query filter params (group.id=4e5ddb7294b24d44a68fd2e6de00c49c, role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:19.632545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:19.635102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:19.646560 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 188/373] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:19 2022] GET /identity/v3/role_assignments?group.id=4e5ddb7294b24d44a68fd2e6de00c49c&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 260 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:19.647817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.688132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b17cc9-07f3-4361-b360-ededfd15529c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:19.730875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5fe08-2766-4e65-bcfb-708dbf66bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:19.731906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5fe08-2766-4e65-bcfb-708dbf66bd2a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:19.732876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5fe08-2766-4e65-bcfb-708dbf66bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:19.733365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5fe08-2766-4e65-bcfb-708dbf66bd2a None admin] PATH_INFO: `/v3/users/267ed5b5eac14f1cab2f370c6aca5b10` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:19.735163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5fe08-2766-4e65-bcfb-708dbf66bd2a None admin] RBAC: Authorizing `identity:get_user(user_id=267ed5b5eac14f1cab2f370c6aca5b10)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:19.736960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5fe08-2766-4e65-bcfb-708dbf66bd2a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:19.739455 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 186/374] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:22:19 2022] GET /identity/v3/users/267ed5b5eac14f1cab2f370c6aca5b10 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:19.747871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5fe08-2766-4e65-bcfb-708dbf66bd2a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.840809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93e26d-8aec-42bf-8489-1667e9be8bf0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:19.884713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:19.892764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:19.893538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:19.895239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:19.896249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:19.897140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:19.898388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:19.898892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:19.900527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:19.912853 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 189/375] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:19.926050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:19.966587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5fe08-2766-4e65-bcfb-708dbf66bd2a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:20.019478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:20.020335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:20.021364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:20.021705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:20.022460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] RBAC: Adding query filter params (role.id=2efac5a86ee940c49657b4c67e2d12e6, user.id=267ed5b5eac14f1cab2f370c6aca5b10, scope.system=all) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:20.022776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:20.024245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:20.042051 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 187/376] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 30 09:22:19 2022] GET /identity/v3/role_assignments?user.id=267ed5b5eac14f1cab2f370c6aca5b10&scope.system=all&role.id=2efac5a86ee940c49657b4c67e2d12e6 => generated 510 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:20.065886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede49da7-d0e0-46e5-88ae-2a8de6589fa6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:20.119133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a8641-6119-46da-bf8b-760dcf2d03ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:20.120214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8641-6119-46da-bf8b-760dcf2d03ce None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:20.120951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8641-6119-46da-bf8b-760dcf2d03ce None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:20.121515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a8641-6119-46da-bf8b-760dcf2d03ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:20.203129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:20.213287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a8641-6119-46da-bf8b-760dcf2d03ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:20.215282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a8641-6119-46da-bf8b-760dcf2d03ce None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:20.329441 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 190/377] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:20.335978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:20.488080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851cd8-cf5a-40cc-9496-1506b624aeb0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:20.488523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df029-b10c-4f34-be37-d62c6619d087 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:20.489343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df029-b10c-4f34-be37-d62c6619d087 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:20.489620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df029-b10c-4f34-be37-d62c6619d087 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:20.491122 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 191/378] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:20.497164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7863d3-a32e-4aab-bf2c-9f750678c347 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:20.497448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7863d3-a32e-4aab-bf2c-9f750678c347 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:20.497820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7863d3-a32e-4aab-bf2c-9f750678c347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:20.517151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:20.532740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bc7863d3-a32e-4aab-bf2c-9f750678c347 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:20.533993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bc7863d3-a32e-4aab-bf2c-9f750678c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:20.535265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306b55f-84df-473e-a127-1f9ec3e730ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:20.536279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306b55f-84df-473e-a127-1f9ec3e730ad None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:20.536757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306b55f-84df-473e-a127-1f9ec3e730ad None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:20.537229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306b55f-84df-473e-a127-1f9ec3e730ad None admin] PATH_INFO: `/v3/projects/9494127904354d908def6993c27761dd` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:20.538604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306b55f-84df-473e-a127-1f9ec3e730ad None admin] RBAC: Authorizing `identity:get_project(project_id=9494127904354d908def6993c27761dd)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:20.540240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306b55f-84df-473e-a127-1f9ec3e730ad None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:20.542079 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 188/379] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:20 2022] GET /identity/v3/projects/9494127904354d908def6993c27761dd => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:20.552663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306b55f-84df-473e-a127-1f9ec3e730ad None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:20.677846 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 192/380] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:20.725923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306b55f-84df-473e-a127-1f9ec3e730ad None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:20.761898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12317d0-2d37-44ba-9d88-68ed479b72a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:20.762866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12317d0-2d37-44ba-9d88-68ed479b72a2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:20.763373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12317d0-2d37-44ba-9d88-68ed479b72a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:20.763838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12317d0-2d37-44ba-9d88-68ed479b72a2 None admin] PATH_INFO: `/v3/groups/15239044fe7c422ea25e72fd012fc22f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:20.770465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12317d0-2d37-44ba-9d88-68ed479b72a2 None admin] RBAC: Authorizing `identity:get_group(group_id=15239044fe7c422ea25e72fd012fc22f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:20.772257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12317d0-2d37-44ba-9d88-68ed479b72a2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:20.774453 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 189/381] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:20 2022] GET /identity/v3/groups/15239044fe7c422ea25e72fd012fc22f => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:20.780410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7863d3-a32e-4aab-bf2c-9f750678c347 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:20.808783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:20.903108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7863d3-a32e-4aab-bf2c-9f750678c347 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:20.945169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:20.945833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:20.946112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:20.946383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:20.947109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] RBAC: Adding query filter params (group.id=15239044fe7c422ea25e72fd012fc22f, role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=9494127904354d908def6993c27761dd) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:20.947481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:20.949044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:20.954864 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 193/382] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:20 2022] GET /identity/v3/role_assignments?group.id=15239044fe7c422ea25e72fd012fc22f&scope.project.id=9494127904354d908def6993c27761dd&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:21.127345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12317d0-2d37-44ba-9d88-68ed479b72a2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:21.197266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:21.251460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12317d0-2d37-44ba-9d88-68ed479b72a2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:21.291701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24ce9e-b4bb-4bf0-8b70-dc42df3b04bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:21.292697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24ce9e-b4bb-4bf0-8b70-dc42df3b04bf None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:21.293176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24ce9e-b4bb-4bf0-8b70-dc42df3b04bf None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:21.293682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24ce9e-b4bb-4bf0-8b70-dc42df3b04bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:21.305517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c24ce9e-b4bb-4bf0-8b70-dc42df3b04bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-9c24ce9e-b4bb-4bf0-8b70-dc42df3b04bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:21.311058 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:21.311919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:21.312698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:21.316650 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 190/383] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:22:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:21.322463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:21.423456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:21.437829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7880cb-5780-4fe8-9ef9-192a3523f663 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:21.477369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:21.478025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:21.478310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:21.478588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:21.479260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:21.479566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:21.480959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:21.493743 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 194/384] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:21.495885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24ce9e-b4bb-4bf0-8b70-dc42df3b04bf None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:21.674336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c24ce9e-b4bb-4bf0-8b70-dc42df3b04bf None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:21.693083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:21.721457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43922bdf-d036-43c3-82a7-ac57e2e224a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:21.722129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43922bdf-d036-43c3-82a7-ac57e2e224a8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:21.722411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43922bdf-d036-43c3-82a7-ac57e2e224a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:21.722686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43922bdf-d036-43c3-82a7-ac57e2e224a8 None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:21.723829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43922bdf-d036-43c3-82a7-ac57e2e224a8 None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:21.725343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43922bdf-d036-43c3-82a7-ac57e2e224a8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:21.727162 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 191/385] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:21 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:21.732877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:21.783405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33f7e-710b-4a7b-9438-34bb7e6d654a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:21.783724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33f7e-710b-4a7b-9438-34bb7e6d654a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:21.784020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33f7e-710b-4a7b-9438-34bb7e6d654a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:21.785703 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 192/386] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:22:21.791759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf210e0d-0de0-43b1-ae01-d8d6b13e9565 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:21.792317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf210e0d-0de0-43b1-ae01-d8d6b13e9565 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:21.792791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf210e0d-0de0-43b1-ae01-d8d6b13e9565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:21.821795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cf210e0d-0de0-43b1-ae01-d8d6b13e9565 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:21.823505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cf210e0d-0de0-43b1-ae01-d8d6b13e9565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:21.907821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac649025-b48d-4bd8-8528-bcd19e10cfb0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:21.924110 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 193/387] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:21.985114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b3916-f758-484d-81e6-138e3b22151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:21.985697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b3916-f758-484d-81e6-138e3b22151d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:21.985981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b3916-f758-484d-81e6-138e3b22151d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:21.986256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b3916-f758-484d-81e6-138e3b22151d None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:21.990343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b3916-f758-484d-81e6-138e3b22151d None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:21.991820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b3916-f758-484d-81e6-138e3b22151d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:21.993832 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 195/388] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:21 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:21.999700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf210e0d-0de0-43b1-ae01-d8d6b13e9565 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:22.154245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf210e0d-0de0-43b1-ae01-d8d6b13e9565 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:22.173016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:22.202110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97eabd-d40e-4a6c-8dc3-afdb8af668e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:22.202784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97eabd-d40e-4a6c-8dc3-afdb8af668e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:22.203119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97eabd-d40e-4a6c-8dc3-afdb8af668e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:22.203397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97eabd-d40e-4a6c-8dc3-afdb8af668e0 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/groups/4e5ddb7294b24d44a68fd2e6de00c49c/roles/89a7fdb1750447479ee3540579dc1e23` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:22.205558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97eabd-d40e-4a6c-8dc3-afdb8af668e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0, group_id=4e5ddb7294b24d44a68fd2e6de00c49c, role_id=89a7fdb1750447479ee3540579dc1e23)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:22.207037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97eabd-d40e-4a6c-8dc3-afdb8af668e0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:22.234437 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 194/389] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 30 09:22:21 2022] PUT /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/groups/4e5ddb7294b24d44a68fd2e6de00c49c/roles/89a7fdb1750447479ee3540579dc1e23 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:22.365748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b3916-f758-484d-81e6-138e3b22151d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:22.388152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:22.480845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b3916-f758-484d-81e6-138e3b22151d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:22.518887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ebdad-a8e2-40ba-8c3f-e58bb1a4101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:22.519594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ebdad-a8e2-40ba-8c3f-e58bb1a4101d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:22.519872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ebdad-a8e2-40ba-8c3f-e58bb1a4101d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:22.520236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ebdad-a8e2-40ba-8c3f-e58bb1a4101d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:22.533242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ebdad-a8e2-40ba-8c3f-e58bb1a4101d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-453ebdad-a8e2-40ba-8c3f-e58bb1a4101d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:22.539192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:22.540104 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:22.540913 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 196/390] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:22.547006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97eabd-d40e-4a6c-8dc3-afdb8af668e0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:22.664081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97eabd-d40e-4a6c-8dc3-afdb8af668e0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:22.701026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e094968-0473-49f9-9249-946c462315aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:22.701745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e094968-0473-49f9-9249-946c462315aa None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:22.702025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e094968-0473-49f9-9249-946c462315aa None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:22.702297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e094968-0473-49f9-9249-946c462315aa None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:22.702966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e094968-0473-49f9-9249-946c462315aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:22.703266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e094968-0473-49f9-9249-946c462315aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:22.704627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e094968-0473-49f9-9249-946c462315aa None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:22.715873 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 195/391] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:22.721588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ebdad-a8e2-40ba-8c3f-e58bb1a4101d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:22.759789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e19d-6cf1-4cf1-8e90-47c78d0f83a7 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:22.760100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e19d-6cf1-4cf1-8e90-47c78d0f83a7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:22.760333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e19d-6cf1-4cf1-8e90-47c78d0f83a7 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:22.761707 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 196/392] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:22.767653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cb2d2-bcbd-4f08-88ce-d77174c1bb96 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:22.767966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cb2d2-bcbd-4f08-88ce-d77174c1bb96 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:22.768191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cb2d2-bcbd-4f08-88ce-d77174c1bb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:22.806729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cb9cb2d2-bcbd-4f08-88ce-d77174c1bb96 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:22.808114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cb9cb2d2-bcbd-4f08-88ce-d77174c1bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:22.819241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:22.864650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ebdad-a8e2-40ba-8c3f-e58bb1a4101d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:22.871392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:22.930263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf5eab-8786-49eb-86d1-da44585e8344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:22.931139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf5eab-8786-49eb-86d1-da44585e8344 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:22.931813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf5eab-8786-49eb-86d1-da44585e8344 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:22.932638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf5eab-8786-49eb-86d1-da44585e8344 None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:22.934223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf5eab-8786-49eb-86d1-da44585e8344 None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:22.936044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf5eab-8786-49eb-86d1-da44585e8344 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:22.937783 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 197/393] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:22.939270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d840f-28fe-44ad-ba63-cb64f8c2a107 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:22.939776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d840f-28fe-44ad-ba63-cb64f8c2a107 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:22.940254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d840f-28fe-44ad-ba63-cb64f8c2a107 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:22.941139 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 197/394] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:22 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:22.942751 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 198/395] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:22.948258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d840f-28fe-44ad-ba63-cb64f8c2a107 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:22.948404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52585f8-1454-44d0-b19c-3f6b11214647 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:22.948541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52585f8-1454-44d0-b19c-3f6b11214647 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:22.949137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52585f8-1454-44d0-b19c-3f6b11214647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:22.983551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a52585f8-1454-44d0-b19c-3f6b11214647 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:22.984748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a52585f8-1454-44d0-b19c-3f6b11214647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:23.093566 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 198/396] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:23.107842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d840f-28fe-44ad-ba63-cb64f8c2a107 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:23.145363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bad043-d57e-4708-8db8-b5b3923535ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:23.145958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bad043-d57e-4708-8db8-b5b3923535ef None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:23.146233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bad043-d57e-4708-8db8-b5b3923535ef None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:23.146500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bad043-d57e-4708-8db8-b5b3923535ef None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:23.147641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bad043-d57e-4708-8db8-b5b3923535ef None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:23.149081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bad043-d57e-4708-8db8-b5b3923535ef None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:23.150752 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 199/397] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:22 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:23.157219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52585f8-1454-44d0-b19c-3f6b11214647 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:23.282089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:23.306032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52585f8-1454-44d0-b19c-3f6b11214647 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:23.362206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e5abd-7bc3-47f1-b65d-39fc16da31c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:23.363229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e5abd-7bc3-47f1-b65d-39fc16da31c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:23.364282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e5abd-7bc3-47f1-b65d-39fc16da31c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:23.364772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e5abd-7bc3-47f1-b65d-39fc16da31c8 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/groups/4e5ddb7294b24d44a68fd2e6de00c49c/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:23.366940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e5abd-7bc3-47f1-b65d-39fc16da31c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0, group_id=4e5ddb7294b24d44a68fd2e6de00c49c, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:23.368592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e5abd-7bc3-47f1-b65d-39fc16da31c8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:23.388161 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 199/398] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 30 09:22:23 2022] PUT /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0/groups/4e5ddb7294b24d44a68fd2e6de00c49c/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:23.436123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bad043-d57e-4708-8db8-b5b3923535ef None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:23.456374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e5abd-7bc3-47f1-b65d-39fc16da31c8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:23.482895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:23.612818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bad043-d57e-4708-8db8-b5b3923535ef None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:23.656510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e5abd-7bc3-47f1-b65d-39fc16da31c8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:23.658039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd252aca-64c3-4b39-a011-68a2eaa91b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:23.658845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd252aca-64c3-4b39-a011-68a2eaa91b9e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:23.659133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd252aca-64c3-4b39-a011-68a2eaa91b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:23.659408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd252aca-64c3-4b39-a011-68a2eaa91b9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:23.674433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd252aca-64c3-4b39-a011-68a2eaa91b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-fd252aca-64c3-4b39-a011-68a2eaa91b9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:23.680714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:23.681590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:23.682833 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 200/399] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:22:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:23.688187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd252aca-64c3-4b39-a011-68a2eaa91b9e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:23.703397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1333ba-bf47-4223-b539-712c16d1f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:23.706691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1333ba-bf47-4223-b539-712c16d1f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:23.707256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1333ba-bf47-4223-b539-712c16d1f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:23.708314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1333ba-bf47-4223-b539-712c16d1f2d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:23.712918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:23.734509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1333ba-bf47-4223-b539-712c16d1f2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1a1333ba-bf47-4223-b539-712c16d1f2d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:23.740230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:23.741190 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:23.742003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:23.743755 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 200/400] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:23.750790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1333ba-bf47-4223-b539-712c16d1f2d4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:23.848785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:23.865254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd252aca-64c3-4b39-a011-68a2eaa91b9e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:23.906620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1333ba-bf47-4223-b539-712c16d1f2d4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:23.907154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdd0fd-1a5b-441b-9b9e-b3f75ec83e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:23.907416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdd0fd-1a5b-441b-9b9e-b3f75ec83e40 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:23.907696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdd0fd-1a5b-441b-9b9e-b3f75ec83e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:23.908241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdd0fd-1a5b-441b-9b9e-b3f75ec83e40 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:23.909842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdd0fd-1a5b-441b-9b9e-b3f75ec83e40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:23.910159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdd0fd-1a5b-441b-9b9e-b3f75ec83e40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:23.912098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdd0fd-1a5b-441b-9b9e-b3f75ec83e40 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:23.925795 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 201/401] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:23.927819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025496a6-5ad7-4589-a4cc-0fcd8d7a7aff None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:23.928105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025496a6-5ad7-4589-a4cc-0fcd8d7a7aff None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:23.928381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025496a6-5ad7-4589-a4cc-0fcd8d7a7aff None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:23.929753 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 202/402] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:23.931248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025496a6-5ad7-4589-a4cc-0fcd8d7a7aff None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:23.958156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78ae0e-e3c5-453c-9df2-c64c5ab56e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:23.959700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78ae0e-e3c5-453c-9df2-c64c5ab56e6a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:23.960215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78ae0e-e3c5-453c-9df2-c64c5ab56e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:23.960690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78ae0e-e3c5-453c-9df2-c64c5ab56e6a None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:23.961558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78ae0e-e3c5-453c-9df2-c64c5ab56e6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:23.962600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78ae0e-e3c5-453c-9df2-c64c5ab56e6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:23.965105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78ae0e-e3c5-453c-9df2-c64c5ab56e6a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:23.975000 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 201/403] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:23.976394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9bbb9-6131-4fa8-a6ff-aab66279b3b1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:23.976755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9bbb9-6131-4fa8-a6ff-aab66279b3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:23.977102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9bbb9-6131-4fa8-a6ff-aab66279b3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:23.996277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-67b9bbb9-6131-4fa8-a6ff-aab66279b3b1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:23.997592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-67b9bbb9-6131-4fa8-a6ff-aab66279b3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:24.058154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:24.073180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025496a6-5ad7-4589-a4cc-0fcd8d7a7aff None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:24.073371 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 202/404] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:24.079320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9bbb9-6131-4fa8-a6ff-aab66279b3b1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:24.146288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e35cb-0544-40e8-8053-71d3b0dd0b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:24.146985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e35cb-0544-40e8-8053-71d3b0dd0b5f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:24.147272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e35cb-0544-40e8-8053-71d3b0dd0b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:24.147547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e35cb-0544-40e8-8053-71d3b0dd0b5f None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:24.150092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e35cb-0544-40e8-8053-71d3b0dd0b5f None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:24.153285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e35cb-0544-40e8-8053-71d3b0dd0b5f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:24.155156 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 203/405] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:23 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:24.160454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e35cb-0544-40e8-8053-71d3b0dd0b5f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:24.191544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:24.260859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9bbb9-6131-4fa8-a6ff-aab66279b3b1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:24.326435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d34ff-02d9-4a04-80db-a0dabf1c96aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:24.327077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d34ff-02d9-4a04-80db-a0dabf1c96aa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:24.327357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d34ff-02d9-4a04-80db-a0dabf1c96aa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:24.327629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d34ff-02d9-4a04-80db-a0dabf1c96aa None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:24.329474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d34ff-02d9-4a04-80db-a0dabf1c96aa None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:24.331627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d34ff-02d9-4a04-80db-a0dabf1c96aa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:24.334068 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 203/406] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:24 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:24.339423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d34ff-02d9-4a04-80db-a0dabf1c96aa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:24.345446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e35cb-0544-40e8-8053-71d3b0dd0b5f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:24.408060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1b44-8cb4-4705-98f2-1b88be984b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:24.408462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1b44-8cb4-4705-98f2-1b88be984b76 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:24.408741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1b44-8cb4-4705-98f2-1b88be984b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:24.409012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1b44-8cb4-4705-98f2-1b88be984b76 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:24.411432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a1b44-8cb4-4705-98f2-1b88be984b76 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:24.420987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a1b44-8cb4-4705-98f2-1b88be984b76 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:24.424377 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 204/407] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:24 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:24.425274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1b44-8cb4-4705-98f2-1b88be984b76 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:24.448206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:24.499059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d34ff-02d9-4a04-80db-a0dabf1c96aa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:24.540387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea8b34-402e-4b75-8e1f-c9dcdc33222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:24.541380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea8b34-402e-4b75-8e1f-c9dcdc33222f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:24.541887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea8b34-402e-4b75-8e1f-c9dcdc33222f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:24.542346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea8b34-402e-4b75-8e1f-c9dcdc33222f None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:24.543210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:24.544626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea8b34-402e-4b75-8e1f-c9dcdc33222f None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:24.546665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea8b34-402e-4b75-8e1f-c9dcdc33222f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:24.550012 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 204/408] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:24 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:24.551382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea8b34-402e-4b75-8e1f-c9dcdc33222f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:24.559053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1b44-8cb4-4705-98f2-1b88be984b76 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:24.604869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771538c-1711-438c-a278-22e1394578be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:24.605664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771538c-1711-438c-a278-22e1394578be None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:24.606110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771538c-1711-438c-a278-22e1394578be None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:24.606484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771538c-1711-438c-a278-22e1394578be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:24.620773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771538c-1711-438c-a278-22e1394578be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-5771538c-1711-438c-a278-22e1394578be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:24.628514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:24.630168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:24.630975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:24.636411 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 205/409] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:22:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:24.636411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771538c-1711-438c-a278-22e1394578be None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:24.690269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea8b34-402e-4b75-8e1f-c9dcdc33222f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:24.721103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:24.736342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cbac8-1bb9-4bc8-992f-5b9bea7b92cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:24.737106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cbac8-1bb9-4bc8-992f-5b9bea7b92cb None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:24.737673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cbac8-1bb9-4bc8-992f-5b9bea7b92cb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:24.737847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cbac8-1bb9-4bc8-992f-5b9bea7b92cb None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/groups/4e5ddb7294b24d44a68fd2e6de00c49c/roles/89a7fdb1750447479ee3540579dc1e23` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:24.741150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cbac8-1bb9-4bc8-992f-5b9bea7b92cb None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e, group_id=4e5ddb7294b24d44a68fd2e6de00c49c, role_id=89a7fdb1750447479ee3540579dc1e23)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:24.742809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cbac8-1bb9-4bc8-992f-5b9bea7b92cb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:24.762971 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 205/410] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 30 09:22:24 2022] PUT /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/groups/4e5ddb7294b24d44a68fd2e6de00c49c/roles/89a7fdb1750447479ee3540579dc1e23 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:24.769735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cbac8-1bb9-4bc8-992f-5b9bea7b92cb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:24.771907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771538c-1711-438c-a278-22e1394578be None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:24.847278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957d980-2541-4e0e-8340-59386aba91bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:24.848341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d980-2541-4e0e-8340-59386aba91bc None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:24.848809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d980-2541-4e0e-8340-59386aba91bc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:24.849280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d980-2541-4e0e-8340-59386aba91bc None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/groups/4e5ddb7294b24d44a68fd2e6de00c49c/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:24.855455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957d980-2541-4e0e-8340-59386aba91bc None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e, group_id=4e5ddb7294b24d44a68fd2e6de00c49c, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:24.857395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957d980-2541-4e0e-8340-59386aba91bc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:24.889215 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 206/411] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 30 09:22:24 2022] PUT /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e/groups/4e5ddb7294b24d44a68fd2e6de00c49c/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:24.906519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cbac8-1bb9-4bc8-992f-5b9bea7b92cb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:24.952588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b5e0f-35f1-4dd2-91cb-f9296fa2bda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:24.953204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b5e0f-35f1-4dd2-91cb-f9296fa2bda9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:24.953468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b5e0f-35f1-4dd2-91cb-f9296fa2bda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:24.953780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b5e0f-35f1-4dd2-91cb-f9296fa2bda9 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:24.954574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b5e0f-35f1-4dd2-91cb-f9296fa2bda9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:24.954905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b5e0f-35f1-4dd2-91cb-f9296fa2bda9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:24.956526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b5e0f-35f1-4dd2-91cb-f9296fa2bda9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:24.967357 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 206/412] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:24.974068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957d980-2541-4e0e-8340-59386aba91bc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:25.001010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:25.035119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b36a-1748-4620-b294-ccb852a05b50 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.035862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b36a-1748-4620-b294-ccb852a05b50 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.036363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b36a-1748-4620-b294-ccb852a05b50 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:25.037997 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 207/413] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:25.043763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f0301-fba5-4e98-bc9e-b53b2b5d5061 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.044337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f0301-fba5-4e98-bc9e-b53b2b5d5061 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.044620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f0301-fba5-4e98-bc9e-b53b2b5d5061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:25.065010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-596f0301-fba5-4e98-bc9e-b53b2b5d5061 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:25.066626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-596f0301-fba5-4e98-bc9e-b53b2b5d5061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:25.128747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:25.139600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957d980-2541-4e0e-8340-59386aba91bc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:25.182935 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 208/414] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:25.196997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba4085-e0f3-4a0c-a15f-66cf3c5f1ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:25.197671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba4085-e0f3-4a0c-a15f-66cf3c5f1ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.197955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba4085-e0f3-4a0c-a15f-66cf3c5f1ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.198228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba4085-e0f3-4a0c-a15f-66cf3c5f1ac6 None admin] PATH_INFO: `/v3/groups/15239044fe7c422ea25e72fd012fc22f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:25.199348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba4085-e0f3-4a0c-a15f-66cf3c5f1ac6 None admin] RBAC: Authorizing `identity:get_group(group_id=15239044fe7c422ea25e72fd012fc22f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:25.200791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba4085-e0f3-4a0c-a15f-66cf3c5f1ac6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:25.202770 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 207/415] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:24 2022] GET /identity/v3/groups/15239044fe7c422ea25e72fd012fc22f => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:25.208265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f0301-fba5-4e98-bc9e-b53b2b5d5061 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:25.327400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f0301-fba5-4e98-bc9e-b53b2b5d5061 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:25.373893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296211e4-1ff7-4a08-95b6-df35f4e418ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:25.374540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296211e4-1ff7-4a08-95b6-df35f4e418ed None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.374823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296211e4-1ff7-4a08-95b6-df35f4e418ed None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.375099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296211e4-1ff7-4a08-95b6-df35f4e418ed None admin] PATH_INFO: `/v3/projects/9494127904354d908def6993c27761dd` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:25.376244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296211e4-1ff7-4a08-95b6-df35f4e418ed None admin] RBAC: Authorizing `identity:get_project(project_id=9494127904354d908def6993c27761dd)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:25.377798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296211e4-1ff7-4a08-95b6-df35f4e418ed None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:25.380219 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 209/416] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:25 2022] GET /identity/v3/projects/9494127904354d908def6993c27761dd => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:25.385912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba4085-e0f3-4a0c-a15f-66cf3c5f1ac6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:25.489567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:25.515587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba4085-e0f3-4a0c-a15f-66cf3c5f1ac6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:25.525762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296211e4-1ff7-4a08-95b6-df35f4e418ed None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:25.559710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62398d30-fbe9-4f07-8ce2-a398dbca569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:25.560735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62398d30-fbe9-4f07-8ce2-a398dbca569b None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.561252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62398d30-fbe9-4f07-8ce2-a398dbca569b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.561753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62398d30-fbe9-4f07-8ce2-a398dbca569b None admin] PATH_INFO: `/v3/projects/9494127904354d908def6993c27761dd/groups/15239044fe7c422ea25e72fd012fc22f/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:25.565080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62398d30-fbe9-4f07-8ce2-a398dbca569b None admin] RBAC: Authorizing `identity:create_grant(project_id=9494127904354d908def6993c27761dd, group_id=15239044fe7c422ea25e72fd012fc22f, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:25.566813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62398d30-fbe9-4f07-8ce2-a398dbca569b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:25.586361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:25.592267 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 208/417] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 30 09:22:25 2022] PUT /identity/v3/projects/9494127904354d908def6993c27761dd/groups/15239044fe7c422ea25e72fd012fc22f/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:25.674357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296211e4-1ff7-4a08-95b6-df35f4e418ed None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:25.723177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c564b44-8ef9-4d74-b733-47cc70c8ffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:25.724338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c564b44-8ef9-4d74-b733-47cc70c8ffb4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.724832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c564b44-8ef9-4d74-b733-47cc70c8ffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.725339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c564b44-8ef9-4d74-b733-47cc70c8ffb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:25.737811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c564b44-8ef9-4d74-b733-47cc70c8ffb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1c564b44-8ef9-4d74-b733-47cc70c8ffb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:25.745288 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:25.746465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:25.747242 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:25.749916 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 210/418] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:22:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:25.758862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62398d30-fbe9-4f07-8ce2-a398dbca569b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:25.886749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62398d30-fbe9-4f07-8ce2-a398dbca569b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:25.926179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803e5ca-9411-4cb0-bbb8-724898a149b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:25.927644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803e5ca-9411-4cb0-bbb8-724898a149b5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.928121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803e5ca-9411-4cb0-bbb8-724898a149b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.928605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803e5ca-9411-4cb0-bbb8-724898a149b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:25.929460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803e5ca-9411-4cb0-bbb8-724898a149b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:25.937989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803e5ca-9411-4cb0-bbb8-724898a149b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:25.939695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803e5ca-9411-4cb0-bbb8-724898a149b5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:25.943108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:25.959104 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 209/419] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:25.965335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c564b44-8ef9-4d74-b733-47cc70c8ffb4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:25.985773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790be5-6dd1-4fcf-8df5-99a858ad9007 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.986383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790be5-6dd1-4fcf-8df5-99a858ad9007 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.986853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790be5-6dd1-4fcf-8df5-99a858ad9007 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:25.988401 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 210/420] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:25.994233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc06405-675e-49d8-b819-5569e114072e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:25.994864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc06405-675e-49d8-b819-5569e114072e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:25.995416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc06405-675e-49d8-b819-5569e114072e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:26.023302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7fc06405-675e-49d8-b819-5569e114072e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:26.024866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7fc06405-675e-49d8-b819-5569e114072e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:26.147089 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 211/421] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:26.159232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c564b44-8ef9-4d74-b733-47cc70c8ffb4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:26.163186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:26.203297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e18f26-e7cf-4e3b-abfc-fa72d346e5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:26.204258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e18f26-e7cf-4e3b-abfc-fa72d346e5be None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:26.204763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e18f26-e7cf-4e3b-abfc-fa72d346e5be None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:26.205320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e18f26-e7cf-4e3b-abfc-fa72d346e5be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:26.279573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e18f26-e7cf-4e3b-abfc-fa72d346e5be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:26.281366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e18f26-e7cf-4e3b-abfc-fa72d346e5be None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:26.411165 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 211/422] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:26.416887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc06405-675e-49d8-b819-5569e114072e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:26.437123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e18f26-e7cf-4e3b-abfc-fa72d346e5be None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:26.441233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:26.536774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc06405-675e-49d8-b819-5569e114072e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:26.570200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:26.591928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ae446-447a-49ac-b92d-dd2e222f8d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:26.592566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ae446-447a-49ac-b92d-dd2e222f8d61 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:26.592843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ae446-447a-49ac-b92d-dd2e222f8d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:26.593112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ae446-447a-49ac-b92d-dd2e222f8d61 None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:26.596825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ae446-447a-49ac-b92d-dd2e222f8d61 None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:26.598794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ae446-447a-49ac-b92d-dd2e222f8d61 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:26.602127 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 212/423] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:26 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:26.610600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ae446-447a-49ac-b92d-dd2e222f8d61 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:26.615676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e18f26-e7cf-4e3b-abfc-fa72d346e5be None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:26.676757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bcbb1-aa97-4c55-ba0d-8e34956ef2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:26.677823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bcbb1-aa97-4c55-ba0d-8e34956ef2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:26.678312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bcbb1-aa97-4c55-ba0d-8e34956ef2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:26.678785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bcbb1-aa97-4c55-ba0d-8e34956ef2e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:26.696013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bcbb1-aa97-4c55-ba0d-8e34956ef2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0e5bcbb1-aa97-4c55-ba0d-8e34956ef2e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:26.701197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:26.702099 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:26.702917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:26.707290 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 212/424] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:26.714535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bcbb1-aa97-4c55-ba0d-8e34956ef2e8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:26.797189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ae446-447a-49ac-b92d-dd2e222f8d61 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:26.824218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:26.864095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290cbf5-bfc2-44b3-a4fe-49c9aa592838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:26.864802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290cbf5-bfc2-44b3-a4fe-49c9aa592838 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:26.865076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290cbf5-bfc2-44b3-a4fe-49c9aa592838 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:26.867341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290cbf5-bfc2-44b3-a4fe-49c9aa592838 None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:26.869487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290cbf5-bfc2-44b3-a4fe-49c9aa592838 None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:26.870949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290cbf5-bfc2-44b3-a4fe-49c9aa592838 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:26.872666 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 213/425] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:26 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:26.879043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290cbf5-bfc2-44b3-a4fe-49c9aa592838 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:26.881168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bcbb1-aa97-4c55-ba0d-8e34956ef2e8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:26.944367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:26.944971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:26.945251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:26.945527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:26.946200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:26.946500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:26.947891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:26.973665 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 213/426] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:26.977024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:27.006189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:27.076147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290cbf5-bfc2-44b3-a4fe-49c9aa592838 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:27.138106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:27.138715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.138989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.139258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.139983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] RBAC: Adding query filter params (group.id=4e5ddb7294b24d44a68fd2e6de00c49c, role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:27.140289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:27.141739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:27.158111 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 214/427] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:26 2022] GET /identity/v3/role_assignments?group.id=4e5ddb7294b24d44a68fd2e6de00c49c&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 611 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:27.165335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404ba85-edcc-46ee-8e63-d86a64ead270 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:27.206854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:27.222490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44621da-09d5-462e-b0a9-6c6c3f8fa31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:27.223689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44621da-09d5-462e-b0a9-6c6c3f8fa31f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.224784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44621da-09d5-462e-b0a9-6c6c3f8fa31f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.225068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44621da-09d5-462e-b0a9-6c6c3f8fa31f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.300550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44621da-09d5-462e-b0a9-6c6c3f8fa31f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:27.302667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44621da-09d5-462e-b0a9-6c6c3f8fa31f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:27.447793 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 214/428] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:27.453447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:27.579330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d26565-c826-4dae-b855-3ed6a21b3eb4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:27.609064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:27.622234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9a233-cd6d-4c07-94b4-c22581e84cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:27.622876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9a233-cd6d-4c07-94b4-c22581e84cad None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.623156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9a233-cd6d-4c07-94b4-c22581e84cad None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.623428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9a233-cd6d-4c07-94b4-c22581e84cad None admin] PATH_INFO: `/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.624561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9a233-cd6d-4c07-94b4-c22581e84cad None admin] RBAC: Authorizing `identity:get_project(project_id=ae2ddd1aac7e4151aae57ef1624fd1b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:27.626071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9a233-cd6d-4c07-94b4-c22581e84cad None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:27.627997 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 215/429] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:27 2022] GET /identity/v3/projects/ae2ddd1aac7e4151aae57ef1624fd1b0 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:27.634032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44621da-09d5-462e-b0a9-6c6c3f8fa31f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:27.653760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593afa8-d99e-498c-ba5c-c6b23344960f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.654048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593afa8-d99e-498c-ba5c-c6b23344960f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.654318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593afa8-d99e-498c-ba5c-c6b23344960f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.655876 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 216/430] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:27.664758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f17289-9857-4bbb-8488-63f896e0d54a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.664993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f17289-9857-4bbb-8488-63f896e0d54a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.665409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f17289-9857-4bbb-8488-63f896e0d54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.670184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:27.684771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-73f17289-9857-4bbb-8488-63f896e0d54a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:27.686095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-73f17289-9857-4bbb-8488-63f896e0d54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:27.758475 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 217/431] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:27.768902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44621da-09d5-462e-b0a9-6c6c3f8fa31f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:27.811845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875a131-d5c3-4c85-ae57-b472c76af426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:27.812481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875a131-d5c3-4c85-ae57-b472c76af426 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.812755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875a131-d5c3-4c85-ae57-b472c76af426 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.813070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875a131-d5c3-4c85-ae57-b472c76af426 None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.814242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875a131-d5c3-4c85-ae57-b472c76af426 None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:27.815680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875a131-d5c3-4c85-ae57-b472c76af426 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:27.821756 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 215/432] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:27 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:27.823848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f17289-9857-4bbb-8488-63f896e0d54a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:27.908139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12c84c-62da-4f55-84c3-97b15bbfa93f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.908800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12c84c-62da-4f55-84c3-97b15bbfa93f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.909292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12c84c-62da-4f55-84c3-97b15bbfa93f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.911336 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 216/433] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:27.916609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcc319-23ab-4b61-b667-4cc41d8d8deb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.917145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcc319-23ab-4b61-b667-4cc41d8d8deb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.917643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcc319-23ab-4b61-b667-4cc41d8d8deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.932373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f17289-9857-4bbb-8488-63f896e0d54a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:27.939578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-94dcc319-23ab-4b61-b667-4cc41d8d8deb None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:27.941482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-94dcc319-23ab-4b61-b667-4cc41d8d8deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:27.967135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:27.977994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:27.978604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:27.978886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:27.979154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:27.979870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] RBAC: Adding query filter params (group.id=4e5ddb7294b24d44a68fd2e6de00c49c, role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:27.980177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:27.981651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:27.989846 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 218/434] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:27 2022] GET /identity/v3/role_assignments?group.id=4e5ddb7294b24d44a68fd2e6de00c49c&scope.project.id=ae2ddd1aac7e4151aae57ef1624fd1b0&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:28.026054 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 217/435] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:28.066471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:28.195246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3b3de-bc47-4735-a4ec-b2e6c2e94841 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:28.229906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416545fe-67a3-4887-ba9c-e596d8672e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:28.230546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416545fe-67a3-4887-ba9c-e596d8672e0b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.230837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416545fe-67a3-4887-ba9c-e596d8672e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.231112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416545fe-67a3-4887-ba9c-e596d8672e0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:28.241789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416545fe-67a3-4887-ba9c-e596d8672e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-416545fe-67a3-4887-ba9c-e596d8672e0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:28.247221 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:28.248102 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:28.248885 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 219/436] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:22:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:28.256296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcc319-23ab-4b61-b667-4cc41d8d8deb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:28.260118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3224d23-c39b-47a4-a658-dd13dedbc2ca None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.260408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3224d23-c39b-47a4-a658-dd13dedbc2ca None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.260683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3224d23-c39b-47a4-a658-dd13dedbc2ca None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:28.263102 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 220/437] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:22:28.268217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f7714-7316-4cc2-98d4-fa3918db2979 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.268500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f7714-7316-4cc2-98d4-fa3918db2979 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.268813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f7714-7316-4cc2-98d4-fa3918db2979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:28.289561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-866f7714-7316-4cc2-98d4-fa3918db2979 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:28.290762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-866f7714-7316-4cc2-98d4-fa3918db2979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:28.291239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:28.350813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:28.376186 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 221/438] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:28.377034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f7714-7316-4cc2-98d4-fa3918db2979 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:28.384305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcc319-23ab-4b61-b667-4cc41d8d8deb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:28.450136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:28.450466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.450725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.450997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:28.451859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:28.452172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:28.455051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:28.465606 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 218/439] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:28.471003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:28.506686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f7714-7316-4cc2-98d4-fa3918db2979 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:28.548536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5dee1-bd0d-4ae9-ab79-e09f1f569b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:28.549264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5dee1-bd0d-4ae9-ab79-e09f1f569b8f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.549539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5dee1-bd0d-4ae9-ab79-e09f1f569b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.549835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5dee1-bd0d-4ae9-ab79-e09f1f569b8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:28.561692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5dee1-bd0d-4ae9-ab79-e09f1f569b8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-01e5dee1-bd0d-4ae9-ab79-e09f1f569b8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:28.568198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:28.569074 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:28.569908 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 222/440] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:28.575621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5dee1-bd0d-4ae9-ab79-e09f1f569b8f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:28.576117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:28.594727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ad22c-d1a3-469f-9a18-2402224da5d2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:28.636573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:28.638743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cf561-64d5-4e7a-a0ba-fbe86ba7e786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:28.639514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cf561-64d5-4e7a-a0ba-fbe86ba7e786 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.639824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cf561-64d5-4e7a-a0ba-fbe86ba7e786 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.640018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cf561-64d5-4e7a-a0ba-fbe86ba7e786 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:28.718846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5dee1-bd0d-4ae9-ab79-e09f1f569b8f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:28.724687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cf561-64d5-4e7a-a0ba-fbe86ba7e786 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:28.726225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cf561-64d5-4e7a-a0ba-fbe86ba7e786 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:28.755924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:28.756940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.757411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.757921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:28.759142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:28.759565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:28.760918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:28.769007 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 223/441] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:22:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:28.770264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:28.814509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:28.831104 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 219/442] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 09:22:28.831935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cf561-64d5-4e7a-a0ba-fbe86ba7e786 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:28.870132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213ca07-841f-43f7-9f57-b79b12f7b288 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:28.871624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:28.904378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c7182-4663-4803-8edd-a7ba2e9486c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:28.904997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c7182-4663-4803-8edd-a7ba2e9486c6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.905271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c7182-4663-4803-8edd-a7ba2e9486c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.905538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c7182-4663-4803-8edd-a7ba2e9486c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:28.915192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c7182-4663-4803-8edd-a7ba2e9486c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-896c7182-4663-4803-8edd-a7ba2e9486c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:28.920349 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:28.921223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:28.922059 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 224/443] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:28.922915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c7182-4663-4803-8edd-a7ba2e9486c6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:28.934112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cf561-64d5-4e7a-a0ba-fbe86ba7e786 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:28.970818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3174b-7f5b-4369-a22f-cf508b850e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:28.971346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3174b-7f5b-4369-a22f-cf508b850e72 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:28.971626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3174b-7f5b-4369-a22f-cf508b850e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:28.971896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3174b-7f5b-4369-a22f-cf508b850e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.022035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c7182-4663-4803-8edd-a7ba2e9486c6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.027886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.041615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3174b-7f5b-4369-a22f-cf508b850e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.043117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3174b-7f5b-4369-a22f-cf508b850e72 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.056223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333953f-723a-40de-87fc-75690e01d458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.056852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333953f-723a-40de-87fc-75690e01d458 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.057142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333953f-723a-40de-87fc-75690e01d458 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.057414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333953f-723a-40de-87fc-75690e01d458 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.058560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333953f-723a-40de-87fc-75690e01d458 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.060004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333953f-723a-40de-87fc-75690e01d458 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.061609 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 225/444] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:28 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:29.066140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333953f-723a-40de-87fc-75690e01d458 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.096768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.141137 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 220/445] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:29.146430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3174b-7f5b-4369-a22f-cf508b850e72 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.171930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333953f-723a-40de-87fc-75690e01d458 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.205288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.205916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.206201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.206473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.207136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:29.208157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.209553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.217425 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 226/446] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:29.218240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.228140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.243938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3174b-7f5b-4369-a22f-cf508b850e72 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.276808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ef32-6b3a-46cd-b2bf-96268baa1add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.277433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ef32-6b3a-46cd-b2bf-96268baa1add None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.277742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ef32-6b3a-46cd-b2bf-96268baa1add None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.278014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381ef32-6b3a-46cd-b2bf-96268baa1add None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.279098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381ef32-6b3a-46cd-b2bf-96268baa1add None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.280547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381ef32-6b3a-46cd-b2bf-96268baa1add None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.282127 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 221/447] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:29.282871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ef32-6b3a-46cd-b2bf-96268baa1add None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.309764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.320130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbc56a-8217-42df-9ef7-522e56b7b046 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.354152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317568f1-24d5-416b-b1a5-ba755eefd308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.354775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317568f1-24d5-416b-b1a5-ba755eefd308 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.355055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317568f1-24d5-416b-b1a5-ba755eefd308 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.355326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317568f1-24d5-416b-b1a5-ba755eefd308 None admin] PATH_INFO: `/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.356429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317568f1-24d5-416b-b1a5-ba755eefd308 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3cd79dd6fb4bca8fe5512dbd160e1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.357915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317568f1-24d5-416b-b1a5-ba755eefd308 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.359516 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 227/448] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/projects/6f3cd79dd6fb4bca8fe5512dbd160e1e => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:29.360350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317568f1-24d5-416b-b1a5-ba755eefd308 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.380643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381ef32-6b3a-46cd-b2bf-96268baa1add None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.416222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc82e4c-5197-484f-bc51-aac2fb8c1145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.416827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc82e4c-5197-484f-bc51-aac2fb8c1145 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.417107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc82e4c-5197-484f-bc51-aac2fb8c1145 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.417442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc82e4c-5197-484f-bc51-aac2fb8c1145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.442946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.465045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317568f1-24d5-416b-b1a5-ba755eefd308 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.486191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc82e4c-5197-484f-bc51-aac2fb8c1145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.487586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc82e4c-5197-484f-bc51-aac2fb8c1145 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.497937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.498541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.498829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.499103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.499827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] RBAC: Adding query filter params (group.id=4e5ddb7294b24d44a68fd2e6de00c49c, role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:29.500138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.501574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.507901 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 228/449] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/role_assignments?group.id=4e5ddb7294b24d44a68fd2e6de00c49c&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:29.508677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.542918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.602519 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 222/450] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:29.608138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc82e4c-5197-484f-bc51-aac2fb8c1145 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.633316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76385-c6bf-4afd-9641-f14a415736a7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.667685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.677333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac282813-0ab8-4831-87a0-933791aa9d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.677781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282813-0ab8-4831-87a0-933791aa9d58 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.678057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282813-0ab8-4831-87a0-933791aa9d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.678713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282813-0ab8-4831-87a0-933791aa9d58 None admin] PATH_INFO: `/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.679484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac282813-0ab8-4831-87a0-933791aa9d58 None admin] RBAC: Authorizing `identity:get_group(group_id=4e5ddb7294b24d44a68fd2e6de00c49c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.680922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac282813-0ab8-4831-87a0-933791aa9d58 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.683471 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 229/451] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/groups/4e5ddb7294b24d44a68fd2e6de00c49c => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:29.688420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac282813-0ab8-4831-87a0-933791aa9d58 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.722322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc82e4c-5197-484f-bc51-aac2fb8c1145 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.763329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b1a36-3b97-47c8-bba2-f3dace895ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.763886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b1a36-3b97-47c8-bba2-f3dace895ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.764160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b1a36-3b97-47c8-bba2-f3dace895ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.764429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b1a36-3b97-47c8-bba2-f3dace895ef4 None admin] PATH_INFO: `/v3/projects/9494127904354d908def6993c27761dd` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.765523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b1a36-3b97-47c8-bba2-f3dace895ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=9494127904354d908def6993c27761dd)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.767047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b1a36-3b97-47c8-bba2-f3dace895ef4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.768656 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 223/452] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/projects/9494127904354d908def6993c27761dd => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:29.774344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b1a36-3b97-47c8-bba2-f3dace895ef4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:29.786865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.796254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac282813-0ab8-4831-87a0-933791aa9d58 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.830512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.831119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.831397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.831668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.832933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] RBAC: Adding query filter params (group.id=4e5ddb7294b24d44a68fd2e6de00c49c, role.id=656b8aa0fc5e4f97b03a4251472f521b, scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:29.833235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.834694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.840161 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 230/453] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/role_assignments?group.id=4e5ddb7294b24d44a68fd2e6de00c49c&scope.project.id=6f3cd79dd6fb4bca8fe5512dbd160e1e&role.id=656b8aa0fc5e4f97b03a4251472f521b => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:29.877248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b1a36-3b97-47c8-bba2-f3dace895ef4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:29.903907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:29.916297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5624ee-999c-47cc-8de3-517c34bc5da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:29.916948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5624ee-999c-47cc-8de3-517c34bc5da4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:29.917227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5624ee-999c-47cc-8de3-517c34bc5da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:29.917500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5624ee-999c-47cc-8de3-517c34bc5da4 None admin] PATH_INFO: `/v3/groups/15239044fe7c422ea25e72fd012fc22f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:29.918694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5624ee-999c-47cc-8de3-517c34bc5da4 None admin] RBAC: Authorizing `identity:get_group(group_id=15239044fe7c422ea25e72fd012fc22f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:29.920167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5624ee-999c-47cc-8de3-517c34bc5da4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:29.921800 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 224/454] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/groups/15239044fe7c422ea25e72fd012fc22f => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:29.927863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:30.033950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451a63e-df55-4003-897c-365eb36d624d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:30.068437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1dd7a-91cd-4292-bec7-dbc91796a2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:30.069047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1dd7a-91cd-4292-bec7-dbc91796a2fb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:30.069327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1dd7a-91cd-4292-bec7-dbc91796a2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:30.069606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1dd7a-91cd-4292-bec7-dbc91796a2fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:30.070352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1dd7a-91cd-4292-bec7-dbc91796a2fb None admin] RBAC: Adding query filter params (group.id=15239044fe7c422ea25e72fd012fc22f, role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=9494127904354d908def6993c27761dd) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:30.070666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1dd7a-91cd-4292-bec7-dbc91796a2fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:30.072109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1dd7a-91cd-4292-bec7-dbc91796a2fb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:30.078050 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 231/455] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 30 09:22:29 2022] GET /identity/v3/role_assignments?group.id=15239044fe7c422ea25e72fd012fc22f&scope.project.id=9494127904354d908def6993c27761dd&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:32.711460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca622-9966-457a-9ca9-5e82c3822e42 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:32.712323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca622-9966-457a-9ca9-5e82c3822e42 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:32.712766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ca622-9966-457a-9ca9-5e82c3822e42 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:32.714535 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 225/456] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:32.721393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04832-3880-44f8-94c0-fb775c5ecc55 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:32.722289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04832-3880-44f8-94c0-fb775c5ecc55 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:32.724100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd04832-3880-44f8-94c0-fb775c5ecc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:32.750646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bfd04832-3880-44f8-94c0-fb775c5ecc55 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:32.752187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bfd04832-3880-44f8-94c0-fb775c5ecc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:32.768663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:32.855351 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 232/457] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:32.938191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798c520-32fe-451d-b832-987362328813 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:32.938511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798c520-32fe-451d-b832-987362328813 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:32.938735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798c520-32fe-451d-b832-987362328813 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:32.940330 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 226/458] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:32.951561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f95b-9acf-45d6-837e-a77d4bd57c4e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:32.951851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f95b-9acf-45d6-837e-a77d4bd57c4e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:32.952129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3f95b-9acf-45d6-837e-a77d4bd57c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:32.983036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-79c3f95b-9acf-45d6-837e-a77d4bd57c4e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:32.984177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-79c3f95b-9acf-45d6-837e-a77d4bd57c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:33.076054 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 233/459] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:33.174277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798c520-32fe-451d-b832-987362328813 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:33.316020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798c520-32fe-451d-b832-987362328813 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:33.367371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:33.368306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:33.368735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:33.369169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:33.383696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:33.385370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:33.396329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfb7c4-c190-4cf3-87fe-228af14ca5c9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:33.396932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfb7c4-c190-4cf3-87fe-228af14ca5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:33.397419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfb7c4-c190-4cf3-87fe-228af14ca5c9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:33.399221 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 234/460] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:33.407515 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:33.408392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:33.409230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:33.409230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:33.409230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:33.409230 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 227/461] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:33.410025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14206cb-b96d-439f-93a1-da65062e3b25 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:33.411802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14206cb-b96d-439f-93a1-da65062e3b25 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:33.412283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14206cb-b96d-439f-93a1-da65062e3b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:33.416613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:33.445339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b14206cb-b96d-439f-93a1-da65062e3b25 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:33.446900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b14206cb-b96d-439f-93a1-da65062e3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:33.481311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:33.523716 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 235/462] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:33.525138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14206cb-b96d-439f-93a1-da65062e3b25 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:33.542150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:33.569266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470f4e2-b5e8-44e6-9f68-4e383d8b8945 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:33.653982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e0c29-8814-4c05-891a-208cef1d4afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:33.655196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e0c29-8814-4c05-891a-208cef1d4afb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:33.656549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e0c29-8814-4c05-891a-208cef1d4afb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:33.657048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e0c29-8814-4c05-891a-208cef1d4afb None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:33.657952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e0c29-8814-4c05-891a-208cef1d4afb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:33.658498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e0c29-8814-4c05-891a-208cef1d4afb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:33.660073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e0c29-8814-4c05-891a-208cef1d4afb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:33.679338 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 228/463] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:33.681060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791953d-8a3f-4ec4-96c1-5a6cddb5c8fa None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:33.681564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791953d-8a3f-4ec4-96c1-5a6cddb5c8fa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:33.682051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791953d-8a3f-4ec4-96c1-5a6cddb5c8fa None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:33.684356 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 229/464] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:33.685474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791953d-8a3f-4ec4-96c1-5a6cddb5c8fa None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:33.696622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14206cb-b96d-439f-93a1-da65062e3b25 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:33.739695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71d0e3-ab07-4c5d-8281-8a57e41a73b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:33.741552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d0e3-ab07-4c5d-8281-8a57e41a73b5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:33.742033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d0e3-ab07-4c5d-8281-8a57e41a73b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:33.742480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71d0e3-ab07-4c5d-8281-8a57e41a73b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:33.757016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71d0e3-ab07-4c5d-8281-8a57e41a73b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:33.759420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71d0e3-ab07-4c5d-8281-8a57e41a73b5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2b71d0e3-ab07-4c5d-8281-8a57e41a73b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:33.780551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:33.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:33.782768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:33.782768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:33.782768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:33.784990 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 236/465] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:33.786743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc854d2-f4ea-469a-a308-12a10e711a10 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:33.787294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc854d2-f4ea-469a-a308-12a10e711a10 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:33.787692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc854d2-f4ea-469a-a308-12a10e711a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:33.812930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0bc854d2-f4ea-469a-a308-12a10e711a10 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:33.813807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791953d-8a3f-4ec4-96c1-5a6cddb5c8fa None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:33.815069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0bc854d2-f4ea-469a-a308-12a10e711a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:33.815733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:33.855852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a0a23-1fcf-441a-8cc4-abaf339a22cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:33.856377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a0a23-1fcf-441a-8cc4-abaf339a22cd None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:33.856663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a0a23-1fcf-441a-8cc4-abaf339a22cd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:33.856937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a0a23-1fcf-441a-8cc4-abaf339a22cd None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:33.857579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a0a23-1fcf-441a-8cc4-abaf339a22cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:33.859059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a0a23-1fcf-441a-8cc4-abaf339a22cd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:33.886191 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 230/466] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:22:33 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:22:33.886690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a0a23-1fcf-441a-8cc4-abaf339a22cd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:33.894556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:33.919288 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 237/467] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:33.920943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc854d2-f4ea-469a-a308-12a10e711a10 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:34.009339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a0a23-1fcf-441a-8cc4-abaf339a22cd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:34.047185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:34.047841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.048119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.048390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.049045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:34.049349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:34.051614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:34.065610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc854d2-f4ea-469a-a308-12a10e711a10 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:34.067415 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 231/468] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:34.072593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:34.109060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:34.110283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.110901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.111389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.123873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:34.125767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:34.135311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:34.147843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:34.148800 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:34.149756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:34.149756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:34.149756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:34.152388 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 238/469] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:34.159410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:34.215580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:34.268962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986c9b8-b554-4167-9ce1-cf05da063198 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:34.283552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54def8-cd10-4dfd-b1c0-774feef0f7ae None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:34.323314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865813a-74dd-4a00-afaf-b1aea7c60166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:34.324350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865813a-74dd-4a00-afaf-b1aea7c60166 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.324646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865813a-74dd-4a00-afaf-b1aea7c60166 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.325092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865813a-74dd-4a00-afaf-b1aea7c60166 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.325816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865813a-74dd-4a00-afaf-b1aea7c60166 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:34.327334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865813a-74dd-4a00-afaf-b1aea7c60166 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:34.333780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:34.334718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.335192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.335650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.336485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:34.337193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:34.339685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:34.355462 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 232/470] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:22:34 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:22:34.356593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865813a-74dd-4a00-afaf-b1aea7c60166 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:34.357683 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 239/471] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:34.363508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:34.475746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:34.502370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514610a-308d-458e-90ec-0a86cdbdfff9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:34.506656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:34.517530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865813a-74dd-4a00-afaf-b1aea7c60166 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:34.544772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cd510-cf16-4de7-8f62-f2da18a404e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:34.545564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd510-cf16-4de7-8f62-f2da18a404e6 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.545974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd510-cf16-4de7-8f62-f2da18a404e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.546344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd510-cf16-4de7-8f62-f2da18a404e6 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.547152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cd510-cf16-4de7-8f62-f2da18a404e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:34.548709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cd510-cf16-4de7-8f62-f2da18a404e6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:34.559801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:34.560421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.560919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.562178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.573847 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 240/472] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:22:34 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:22:34.574772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:34.575912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff526f-37e1-4b72-8425-3753e5590274 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.576199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff526f-37e1-4b72-8425-3753e5590274 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.576477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff526f-37e1-4b72-8425-3753e5590274 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.582905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:34.583099 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 241/473] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:34.584703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879a84-2acb-4b01-bf2c-14b4a69ade7f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.585002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879a84-2acb-4b01-bf2c-14b4a69ade7f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.585264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879a84-2acb-4b01-bf2c-14b4a69ade7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.607020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-31879a84-2acb-4b01-bf2c-14b4a69ade7f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:34.607318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:34.608165 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:34.608988 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:34.608988 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:34.608988 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:34.609156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-31879a84-2acb-4b01-bf2c-14b4a69ade7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:34.609231 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 233/474] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:34.616178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:34.712277 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 242/475] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:34.748555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081548e6-dabc-4aaf-a88d-ab462b553688 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:34.780270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:34.789137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:34.789800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.790392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.790663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.791327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:34.791651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:34.793053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:34.806447 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 234/476] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:34.813028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31879a84-2acb-4b01-bf2c-14b4a69ade7f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:34.942733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:34.952422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31879a84-2acb-4b01-bf2c-14b4a69ade7f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:34.990919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f0935-8733-417d-9ccc-3cefefdc8e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:34.991538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f0935-8733-417d-9ccc-3cefefdc8e21 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:34.991817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f0935-8733-417d-9ccc-3cefefdc8e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:34.992200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f0935-8733-417d-9ccc-3cefefdc8e21 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:34.992738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f0935-8733-417d-9ccc-3cefefdc8e21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:34.994235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f0935-8733-417d-9ccc-3cefefdc8e21 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:35.009768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:35.029046 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 243/477] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:22:34 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:22:35.160757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358dd3e-8455-45c2-b54e-d69532f5cbdc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:35.205602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:35.206746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:35.207819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:35.208305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:35.209245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:35.211147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:35.223278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.sql.core [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 30 09:22:35.223278 dv-cinder-master-3-smb devstack@keystone.service[81371]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 30 09:22:35.223278 dv-cinder-master-3-smb devstack@keystone.service[81371]: [parameters: {'id': '97b01e7b15ac46c3ba58bb2d6e760bec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 30 09:22:35.223278 dv-cinder-master-3-smb devstack@keystone.service[81371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Sep 30 09:22:35.244458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return ref.to_dict() Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 09:22:35.245451 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application yield resource Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.rollback() Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.force_reraise() Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise self.value Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application session.commit() Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 30 09:22:35.246395 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._prepare_impl() Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.flush() Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._flush(objects) Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application compat.raise_( Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application flush_context.execute() Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rec.execute(self) Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 30 09:22:35.247245 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 09:22:35.248098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 09:22:35.248923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [parameters: {'id': '97b01e7b15ac46c3ba58bb2d6e760bec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:35.255605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 30 09:22:35.257600 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:35.261379 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 235/478] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 30 09:22:35 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:35.267645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f0935-8733-417d-9ccc-3cefefdc8e21 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:35.413887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f0935-8733-417d-9ccc-3cefefdc8e21 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:35.451089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03bccc-e024-4144-b579-92e8953880fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:35.451792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03bccc-e024-4144-b579-92e8953880fe None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:35.452069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03bccc-e024-4144-b579-92e8953880fe None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:35.452339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03bccc-e024-4144-b579-92e8953880fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:35.463698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03bccc-e024-4144-b579-92e8953880fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ec03bccc-e024-4144-b579-92e8953880fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:35.468879 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:35.469765 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:35.470552 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 244/479] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:22:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:35.477542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:35.534351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:35.591243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786c10d-7691-4953-b804-6f0a15a1aa3c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:35.627461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ea5c7-8ff0-4b92-8f10-69f1c1ef509c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:35.628418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ea5c7-8ff0-4b92-8f10-69f1c1ef509c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:35.629487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ea5c7-8ff0-4b92-8f10-69f1c1ef509c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:35.630000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ea5c7-8ff0-4b92-8f10-69f1c1ef509c None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:35.631020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ea5c7-8ff0-4b92-8f10-69f1c1ef509c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:35.631522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ea5c7-8ff0-4b92-8f10-69f1c1ef509c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:35.633107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ea5c7-8ff0-4b92-8f10-69f1c1ef509c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:35.645738 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 236/480] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:36.674851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe5c8-8fe6-423e-9ade-a9c71296210f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:36.675339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe5c8-8fe6-423e-9ade-a9c71296210f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:36.675339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fe5c8-8fe6-423e-9ade-a9c71296210f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:36.678214 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 245/481] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:36.683115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a46d8-32c2-48b7-b57c-afc147b6a972 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:36.683524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a46d8-32c2-48b7-b57c-afc147b6a972 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:36.683686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a46d8-32c2-48b7-b57c-afc147b6a972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:36.706570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-013a46d8-32c2-48b7-b57c-afc147b6a972 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:36.708848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-013a46d8-32c2-48b7-b57c-afc147b6a972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:36.807788 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 237/482] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:37.167098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fe5c8-8fe6-423e-9ade-a9c71296210f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:37.177091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:37.302402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fe5c8-8fe6-423e-9ade-a9c71296210f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:37.350868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e858cc1-a3be-4392-8604-b421fa247da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:37.351837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858cc1-a3be-4392-8604-b421fa247da9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:37.352336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858cc1-a3be-4392-8604-b421fa247da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:37.352812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e858cc1-a3be-4392-8604-b421fa247da9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:37.370493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e858cc1-a3be-4392-8604-b421fa247da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-7e858cc1-a3be-4392-8604-b421fa247da9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:37.376141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:37.377025 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:37.377826 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:37.380765 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 246/483] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:37.387089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a46d8-32c2-48b7-b57c-afc147b6a972 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:37.446172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:37.503277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a46d8-32c2-48b7-b57c-afc147b6a972 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:37.514577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43097d-8dc0-4828-b269-6c232d323fae None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:37.515159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43097d-8dc0-4828-b269-6c232d323fae None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:37.515626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43097d-8dc0-4828-b269-6c232d323fae None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:37.517382 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 247/484] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:22:37.523551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deddb1-298b-4fff-9715-b359943328d8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:37.524068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deddb1-298b-4fff-9715-b359943328d8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:37.524557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deddb1-298b-4fff-9715-b359943328d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:37.539195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:37.539808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:37.540088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:37.540357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:37.541020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:37.541311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:37.542734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:37.545577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f0deddb1-298b-4fff-9715-b359943328d8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:37.546991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f0deddb1-298b-4fff-9715-b359943328d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:37.551762 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 238/485] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:37.557108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:37.621306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:37.637544 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 248/486] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:37.717478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e592be-70c7-4a46-b801-729b96f88133 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:37.720765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:37.757083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472dfb5-c206-4cd1-9c09-75416da98131 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:37.757586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472dfb5-c206-4cd1-9c09-75416da98131 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:37.757704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472dfb5-c206-4cd1-9c09-75416da98131 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:37.760435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:37.760587 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 249/487] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:22:37.760666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:37.760666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:37.760666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:37.765697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b65c0e-6dd2-4782-9cfa-7ee947ae171d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:37.766047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b65c0e-6dd2-4782-9cfa-7ee947ae171d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:37.766384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b65c0e-6dd2-4782-9cfa-7ee947ae171d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:37.777517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:37.779204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:37.792724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-72b65c0e-6dd2-4782-9cfa-7ee947ae171d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:37.793994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-72b65c0e-6dd2-4782-9cfa-7ee947ae171d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:37.811956 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:37.812903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:37.814369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:37.814369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:37.814369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:37.814369 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 239/488] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:37.819196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:37.939345 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 250/489] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:37.972148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca1bdc-aa0a-4468-8774-e5ef77a6f89d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:38.016804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:38.017422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.017720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.017987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.018632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:38.018930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:38.020384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:38.029659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b65c0e-6dd2-4782-9cfa-7ee947ae171d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.036333 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 240/490] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:38.044245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.080388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:38.203528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b65c0e-6dd2-4782-9cfa-7ee947ae171d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:38.228385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe54aec-2333-4043-8907-d8360219c4c8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:38.233479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:38.263317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35851f-2415-4a51-af67-c27bcc5bedb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:38.263962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35851f-2415-4a51-af67-c27bcc5bedb4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.264239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35851f-2415-4a51-af67-c27bcc5bedb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.264509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35851f-2415-4a51-af67-c27bcc5bedb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.269559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:38.270309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.270588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.270864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.278748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:38.280241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:38.293650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35851f-2415-4a51-af67-c27bcc5bedb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:38.295131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:38.296014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:38.297302 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 241/491] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:38.297912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3c35851f-2415-4a51-af67-c27bcc5bedb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:38.302893 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:38.303790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:38.305471 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 251/492] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:38.310834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35851f-2415-4a51-af67-c27bcc5bedb4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.384529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:38.415004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83127249-8a30-4506-bae4-97a8689144dd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:38.428834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35851f-2415-4a51-af67-c27bcc5bedb4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:38.454767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206acb14-7d3d-4c2d-bf2b-eb3aa3d4d38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:38.455507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206acb14-7d3d-4c2d-bf2b-eb3aa3d4d38c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.455783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206acb14-7d3d-4c2d-bf2b-eb3aa3d4d38c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.456049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206acb14-7d3d-4c2d-bf2b-eb3aa3d4d38c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.463022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c3eed-0a07-4a24-90c2-11b814df0bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:38.463620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c3eed-0a07-4a24-90c2-11b814df0bee None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.463897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c3eed-0a07-4a24-90c2-11b814df0bee None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.464157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c3eed-0a07-4a24-90c2-11b814df0bee None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.464876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c3eed-0a07-4a24-90c2-11b814df0bee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:38.465172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c3eed-0a07-4a24-90c2-11b814df0bee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:38.466637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c3eed-0a07-4a24-90c2-11b814df0bee None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:38.467591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206acb14-7d3d-4c2d-bf2b-eb3aa3d4d38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-206acb14-7d3d-4c2d-bf2b-eb3aa3d4d38c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:38.473904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:38.474769 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:38.475553 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 242/493] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:38.476200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206acb14-7d3d-4c2d-bf2b-eb3aa3d4d38c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.504681 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 252/494] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:38.505988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c6d1-0d7b-49a7-95df-3f1d6547dbc2 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.506282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c6d1-0d7b-49a7-95df-3f1d6547dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.506602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8c6d1-0d7b-49a7-95df-3f1d6547dbc2 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.507949 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 253/495] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:38.508759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c323d-73b8-4b1b-b6bc-69485957a5d6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.509035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c323d-73b8-4b1b-b6bc-69485957a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.509302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c323d-73b8-4b1b-b6bc-69485957a5d6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.510502 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 254/496] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:38.511179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c323d-73b8-4b1b-b6bc-69485957a5d6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.558695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:38.580485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206acb14-7d3d-4c2d-bf2b-eb3aa3d4d38c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:38.612833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:38.616600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.616600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.616600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.616600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:38.616600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:38.616995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:38.618079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c323d-73b8-4b1b-b6bc-69485957a5d6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:38.619230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:38.625947 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 243/497] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:38.626760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.661019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbc6e9-c588-42ea-8f0b-7c7d7550e23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:38.661616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbc6e9-c588-42ea-8f0b-7c7d7550e23f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.661925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbc6e9-c588-42ea-8f0b-7c7d7550e23f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.662203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbc6e9-c588-42ea-8f0b-7c7d7550e23f None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.662853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbc6e9-c588-42ea-8f0b-7c7d7550e23f None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:38.663149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbc6e9-c588-42ea-8f0b-7c7d7550e23f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:38.664511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbc6e9-c588-42ea-8f0b-7c7d7550e23f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:38.672228 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 255/498] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:38.673433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b346a4-3943-42f1-b99a-538f18709f44 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.676476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b346a4-3943-42f1-b99a-538f18709f44 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.676770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b346a4-3943-42f1-b99a-538f18709f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.695270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-15b346a4-3943-42f1-b99a-538f18709f44 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:38.696483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-15b346a4-3943-42f1-b99a-538f18709f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:38.726306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348232c3-6bd5-4d9c-ba25-dd7037abc36f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:38.759482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9b2b-686f-4987-b8e9-5c9340174545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:38.760008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9b2b-686f-4987-b8e9-5c9340174545 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.760286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9b2b-686f-4987-b8e9-5c9340174545 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.760556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9b2b-686f-4987-b8e9-5c9340174545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.761952 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 256/499] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:38.774241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdbfa-4093-4fcb-a046-dc870d6e1e39 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:38.774241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdbfa-4093-4fcb-a046-dc870d6e1e39 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:38.774241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfdbfa-4093-4fcb-a046-dc870d6e1e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:38.781517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-42bfdbfa-4093-4fcb-a046-dc870d6e1e39 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:38.782718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-42bfdbfa-4093-4fcb-a046-dc870d6e1e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:38.815035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:38.829349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:38.834414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c9b2b-686f-4987-b8e9-5c9340174545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:38.835929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c9b2b-686f-4987-b8e9-5c9340174545 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:38.865403 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 257/500] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:38.866321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfdbfa-4093-4fcb-a046-dc870d6e1e39 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.967355 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 244/501] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:38.967355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9b2b-686f-4987-b8e9-5c9340174545 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:38.988647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfdbfa-4093-4fcb-a046-dc870d6e1e39 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.026842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.027837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.028328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.028808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.041759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.044143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:39.060438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:39.061385 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:39.062246 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:39.062246 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:39.062246 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:39.063101 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 258/502] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:39.063808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.078214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.083005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9b2b-686f-4987-b8e9-5c9340174545 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.089162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.120077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.120704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.120977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.121247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.131941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.133394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:39.147939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:39.148959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:39.149793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:39.149793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:39.149793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:39.149793 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 245/503] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:39.150019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.179088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ba25d-9c5e-4c56-a967-ace82e4b70eb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.211704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.212326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.212608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.212879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.223036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.224484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:39.238713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:39.239754 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:39.240677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:39.240677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:39.240677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:39.240677 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 259/504] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:39.240677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.252124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b501e-60d2-4b39-a645-50c3f4ee22ae None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.289716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f55d4-d640-40ba-b573-308b25c9dbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.290295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f55d4-d640-40ba-b573-308b25c9dbca None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.290576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f55d4-d640-40ba-b573-308b25c9dbca None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.292903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f55d4-d640-40ba-b573-308b25c9dbca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.303800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f55d4-d640-40ba-b573-308b25c9dbca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-ff1f55d4-d640-40ba-b573-308b25c9dbca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:39.309058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:39.310105 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:39.310921 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 246/505] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:39.311549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f55d4-d640-40ba-b573-308b25c9dbca None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.324117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.324954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.356873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b710e8-5a5d-4e38-9298-c6f775df063b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.389721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.390371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.390660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.390938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.391578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:39.391881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.393305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.406273 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 260/506] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:39.407140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.417312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f55d4-d640-40ba-b573-308b25c9dbca None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.451220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.451844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.452177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.452447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.463254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.464711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:39.478392 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:39.479398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:39.480222 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:39.480222 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:39.480222 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:39.480222 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 247/507] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:39.480629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.507717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f0c14-8ee2-45a0-b9a2-be1e1c90b201 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.535485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.541647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.542186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.542461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.542733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.543361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:39.543661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.543757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.545033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.556603 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 261/508] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:39.557390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.583536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44e5f0-0c41-48c3-95cb-aca9cd5b9fe1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.617851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.618460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.618739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.619021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.619656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:39.619960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.621358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.633253 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 248/509] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:39.634088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.657447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab3483-51d9-4021-820f-732836b6a64d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.691058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.692499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.692770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.693035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.693713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:39.694012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.695435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.703469 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 262/510] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:39.704244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.733232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aab511-6ccd-490f-9d12-5f3cf7418f0d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.755894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.756506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.767784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.768405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.768680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.768946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.776561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.778045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:39.789720 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:39.790590 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:39.791448 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 249/511] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:39.792043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.805885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93ed41-07b9-444a-87b4-b7a16e6e9f25 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.839235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.839839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.840119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.840395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.841034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:39.841338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.842743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.855856 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 263/512] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:39.856709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.891476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c175d0-e1e0-43e3-8c92-3758b8505cbf None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.923906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.924519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.924788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.925056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:39.932292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:39.933792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:39.944903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:39.946014 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 250/513] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:39.947233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:39.960171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc15dea-b5e3-4731-bb91-ce2fdcdce9a3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:39.973511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.990850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:39.996313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:39.997045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:39.997323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:39.997600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.004887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.006349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:40.017977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:40.018813 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:40.019866 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 264/514] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:40.020332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.051421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968598-9ade-4ce6-83fe-51845088820f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.084036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.084647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.084922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.085191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.095183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.096620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:40.110697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:40.111639 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:40.112527 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:40.112527 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:40.112527 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:40.112527 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 251/515] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:40.112527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.122964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13712b-57fb-424c-97d5-492032bdbabe None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.157461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.158507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.158994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.159473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.160431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:40.160740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.162221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.179045 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 265/516] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:40.179846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.209184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:40.210573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:40.217140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1395a5-e197-4fd7-8504-574d7997c582 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.250916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f39842-4ba6-4e39-b69c-b261db10a780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.251523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f39842-4ba6-4e39-b69c-b261db10a780 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.251807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f39842-4ba6-4e39-b69c-b261db10a780 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.252085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f39842-4ba6-4e39-b69c-b261db10a780 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.252735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f39842-4ba6-4e39-b69c-b261db10a780 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.254838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f39842-4ba6-4e39-b69c-b261db10a780 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.275163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 252/517] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:22:40 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:22:40.275966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f39842-4ba6-4e39-b69c-b261db10a780 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.293168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca006c0-4af5-44c3-9a1c-683fea874e89 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.337044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.337588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.337976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.338255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.339026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:40.339336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.340843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.370628 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 266/518] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:40.370628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.402774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f39842-4ba6-4e39-b69c-b261db10a780 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.442640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.443363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.443647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.443917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.444622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:40.444925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.446440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.466561 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 253/519] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:40.466979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.481941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:40.485130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:40.501888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc4f25-4e76-4657-afc6-3a024a1ee4cd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.539322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.539933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.540212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.540486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.541124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:40.541427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.542846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.559129 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 267/520] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:40.559129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.585734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cfd1b-0abc-433b-8f7f-9a4a25cdc51a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.621333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5c1bd-753c-4f7d-ba14-6dbcd9ee3b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.621964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5c1bd-753c-4f7d-ba14-6dbcd9ee3b3d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.622242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5c1bd-753c-4f7d-ba14-6dbcd9ee3b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.622509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5c1bd-753c-4f7d-ba14-6dbcd9ee3b3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.659201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eabbc0-80d1-445a-a53b-62fb0fa8678e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.693153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5c1bd-753c-4f7d-ba14-6dbcd9ee3b3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.696176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600050d-5770-489f-bd37-f85268226105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.696302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5c1bd-753c-4f7d-ba14-6dbcd9ee3b3d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.696405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600050d-5770-489f-bd37-f85268226105 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.696405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600050d-5770-489f-bd37-f85268226105 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.696405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600050d-5770-489f-bd37-f85268226105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.706520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:40.719000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:40.771743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600050d-5770-489f-bd37-f85268226105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.773301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600050d-5770-489f-bd37-f85268226105 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.804235 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 254/521] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:40.809220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5c1bd-753c-4f7d-ba14-6dbcd9ee3b3d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.876791 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 268/522] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:40.877385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600050d-5770-489f-bd37-f85268226105 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.916581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5c1bd-753c-4f7d-ba14-6dbcd9ee3b3d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:40.928009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:40.932737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:40.958560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:40.959166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:40.959443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:40.959710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:40.960848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=aa055ee122ce4e92b4da7f8222521a9e) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:40.961155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:40.963001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:40.969310 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 255/523] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/role_assignments?user.id=aa055ee122ce4e92b4da7f8222521a9e&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:40.970135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:40.992739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600050d-5770-489f-bd37-f85268226105 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.033132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.033347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.033588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.033883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.044752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.046684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:41.068898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:41.070035 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 09:22:41.070920 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:41.078227 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 269/524] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:41.078227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:41.121610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec788d1-cf63-412b-ba73-d45fc9d13a46 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.161497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.162463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.162918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.163359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.176218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.177822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:41.187350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080d532-ba38-470f-86a8-0bc8cf980729 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:41.194812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:41.195923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:41.197068 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:41.197068 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:41.197068 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:41.201733 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 256/525] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:41.202801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:41.205796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:41.238724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:41.249998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.250908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.251398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.251888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.265466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.267937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:41.286542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:41.287494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:41.288341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:41.288341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:41.288341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:41.290665 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 270/526] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:41.295913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:41.331295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324944f-3600-4e1f-94ea-edcfec658182 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.376491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.377947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.378229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.378498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.379864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:41.380173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.381659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:41.417310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce19d4-6e6b-479e-a339-11d81c13faf9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.436220 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 257/527] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 30 09:22:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:41.437572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:41.461302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.462229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.462511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.462784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.463427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:41.463728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.465374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:41.481711 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 271/528] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:41.482892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:41.514597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:41.559123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:41.578559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a9ece-abd0-44a4-8198-c5fe9e9a5f9f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.609411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04027c5e-6df4-4647-a145-3162dfbe5280 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.615512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.616130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.616415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.616691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.617401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:41.617737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.619141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:41.636251 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 258/529] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:41.636980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:41.651915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c9371-1be7-4c88-b48b-b9e28ddc7621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.652867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c9371-1be7-4c88-b48b-b9e28ddc7621 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.653367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c9371-1be7-4c88-b48b-b9e28ddc7621 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.653915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c9371-1be7-4c88-b48b-b9e28ddc7621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.734313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c9371-1be7-4c88-b48b-b9e28ddc7621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.736096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c9371-1be7-4c88-b48b-b9e28ddc7621 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:41.741035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3538-236c-4e29-980e-4f73593b3c21 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.760864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:41.777112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.777798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.778100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.778993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.786123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.787895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:41.799821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:41.801433 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:41.803145 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:41.803145 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:41.803145 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:41.803145 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:41.803145 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:41.803145 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:41.803145 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:41.803145 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:41.806965 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 259/530] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:41.807904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:41.849543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:41.856901 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 272/531] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:41.858053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c9371-1be7-4c88-b48b-b9e28ddc7621 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:41.924800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520c784-205d-4bfd-8fb9-75ef06f1f069 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.969613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:41.970880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:41.971411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:41.972080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:41.982084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c9371-1be7-4c88-b48b-b9e28ddc7621 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:41.982791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:41.985648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:42.002874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:42.004384 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:42.005223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:42.005223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:42.005223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:42.005223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:42.005223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:42.005223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:42.005223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:42.005223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:42.007020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:42.009245 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 260/532] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:42.010328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:42.027925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:42.028538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.028819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.029093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.029818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:42.030123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:42.031567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.056977 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 273/533] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:42.057957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:42.093703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:42.141421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fabac-baa8-464f-b3f5-623e7c812966 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:42.168555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1296147-4df7-4731-9831-973b556997a3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:42.180006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:42.181047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.181522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.182006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.195466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:42.197412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.205217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:42.205852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.206132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.206401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.207291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:42.207976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:42.209401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:42.214614 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:42.215493 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:42.216303 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:42.216303 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:42.216303 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:42.218420 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 261/534] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:42.219389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:42.226410 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 274/535] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:42.227245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:42.278133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:42.331189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956a21e-4d44-4dfa-aed7-ef68dbd00385 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:42.340119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67cb5e-aa33-484b-8090-d8f0936cd945 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:42.350549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:42.370854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:42.371866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.372389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.372878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.373841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=f41e236339784d169258f06f1090b7f6) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:42.374414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:42.376212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.377124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:42.377765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.378049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.378325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.379693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:42.380221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:42.381657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.384922 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 262/536] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:42 2022] GET /identity/v3/role_assignments?user.id=f41e236339784d169258f06f1090b7f6&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:42.386079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:42.418633 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 275/537] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:42.423704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:42.508080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be06fc0-d795-4247-a059-e47aeb4cdca0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:42.546446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262ffbf-88d0-4e32-8c6a-830ee46c7cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:42.547378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262ffbf-88d0-4e32-8c6a-830ee46c7cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.547853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262ffbf-88d0-4e32-8c6a-830ee46c7cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.548440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262ffbf-88d0-4e32-8c6a-830ee46c7cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.549366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262ffbf-88d0-4e32-8c6a-830ee46c7cc4 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=7fe7a090bd7941bb827d301aa4949857) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:42.549913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262ffbf-88d0-4e32-8c6a-830ee46c7cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:42.552207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262ffbf-88d0-4e32-8c6a-830ee46c7cc4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.558971 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 263/538] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:42 2022] GET /identity/v3/role_assignments?user.id=7fe7a090bd7941bb827d301aa4949857&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:42.571216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388e5d-c0a6-403a-a089-8462756acaef None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.571729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388e5d-c0a6-403a-a089-8462756acaef None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.572147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388e5d-c0a6-403a-a089-8462756acaef None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.573827 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 264/539] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:42.583370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb97d8-3c4f-4327-b567-0411c45098ae None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.583742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb97d8-3c4f-4327-b567-0411c45098ae None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.583911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb97d8-3c4f-4327-b567-0411c45098ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.609375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:42.616355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567caa7-f4e2-48c4-9d49-26a560e56901 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:42.619583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7bbb97d8-3c4f-4327-b567-0411c45098ae None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:42.624051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7bbb97d8-3c4f-4327-b567-0411c45098ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:42.708362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:42.713406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.713735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.713983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.715197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:42.725873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:42.727383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:42.747975 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:42.748894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:42.750429 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 276/540] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:42.755782 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 265/541] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:42.756837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:42.867062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e90a7e-5ffb-41a7-a655-56a9e1cb8f9a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:42.902471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:42.903082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:42.903360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:42.903635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:42.904323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:42.904684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:42.906155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:42.919072 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 277/542] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:42.924591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb97d8-3c4f-4327-b567-0411c45098ae None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:43.063846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb97d8-3c4f-4327-b567-0411c45098ae None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:43.110912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:43.111530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:43.111809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:43.112084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:43.112823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=2c1784d615d8439a9a4cb9fea305a41b) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:43.113173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:43.114648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:43.120860 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 266/543] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:42 2022] GET /identity/v3/role_assignments?user.id=2c1784d615d8439a9a4cb9fea305a41b&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:43.150547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:43.191780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:43.307864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d585a9a-f6b7-4619-8fa2-e76c45e4acf4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:43.349979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771da524-0f7b-4d43-9166-ae3530f9f6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:43.350622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da524-0f7b-4d43-9166-ae3530f9f6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:43.350901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da524-0f7b-4d43-9166-ae3530f9f6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:43.351175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da524-0f7b-4d43-9166-ae3530f9f6d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:43.365612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771da524-0f7b-4d43-9166-ae3530f9f6d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-771da524-0f7b-4d43-9166-ae3530f9f6d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:43.370432 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:43.372657 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:43.374666 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 278/544] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:43.378277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:43.384316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ce0ce-c496-4e67-8391-b243b72d25a0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:43.384316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ce0ce-c496-4e67-8391-b243b72d25a0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:43.384316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ce0ce-c496-4e67-8391-b243b72d25a0 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:43.384316 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 279/545] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:22:43.391290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae593-b391-49fd-b3eb-1c3f9ef30e9c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:43.391870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae593-b391-49fd-b3eb-1c3f9ef30e9c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:43.392335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae593-b391-49fd-b3eb-1c3f9ef30e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:43.412602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e18ae593-b391-49fd-b3eb-1c3f9ef30e9c None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:43.414264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e18ae593-b391-49fd-b3eb-1c3f9ef30e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:43.469768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:43.537654 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 280/546] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:43.555535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49187ebe-f464-4043-a3e9-aac0cc35b634 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:43.624417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:43.624968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:43.625251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:43.625523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:43.626247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:43.626550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:43.627990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:43.646782 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 267/547] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:43.652746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ae593-b391-49fd-b3eb-1c3f9ef30e9c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:43.668973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:43.782071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ae593-b391-49fd-b3eb-1c3f9ef30e9c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:43.824622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:43.825255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:43.825532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:43.825874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:43.841617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:43.843081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:43.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:43.862220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:43.863630 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:43.863630 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:43.863630 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:43.863630 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 281/548] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:43.868078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:43.899281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:43.965054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:44.009677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f660c1-b49b-44ae-b12e-cdec3945cf74 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:44.046806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125ef29-39c0-48d2-af3d-e4f1bdc9a4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:44.047890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125ef29-39c0-48d2-af3d-e4f1bdc9a4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:44.048447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125ef29-39c0-48d2-af3d-e4f1bdc9a4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:44.048912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125ef29-39c0-48d2-af3d-e4f1bdc9a4a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:44.061078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125ef29-39c0-48d2-af3d-e4f1bdc9a4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:44.063635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d125ef29-39c0-48d2-af3d-e4f1bdc9a4a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:44.069200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:44.070615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:44.071590 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:44.072212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87f643-bfd2-4b9f-9a60-fd2d7ba2d8b7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:44.077987 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 282/549] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:44.080236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125ef29-39c0-48d2-af3d-e4f1bdc9a4a9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:44.121382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:44.122115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:44.122466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:44.122989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:44.123791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:44.124166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:44.125892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:44.140248 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 268/550] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:44.145313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:44.256742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125ef29-39c0-48d2-af3d-e4f1bdc9a4a9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:44.288591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:44.308200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:44.308817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:44.309100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:44.309372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:44.310138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:44.310441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:44.311813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:44.318494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02892361-eef7-4d4e-9c12-e1fe11b11f48 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:44.326958 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 283/551] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:44.333172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:44.355747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:44.356378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:44.356655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:44.356928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:44.364534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:44.366012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:44.379346 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:44.380640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:44.381545 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 269/552] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:22:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:44.386539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:44.430858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:44.516953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d1b11-02a1-4f65-b559-1017f94c2d24 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:44.563215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:44.563874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:44.564156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:44.564428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:44.578021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:44.579557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:44.590661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e26fad-7509-4e34-a266-4daefa71c024 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:44.596015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:44.597849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:44.598905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:44.598905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:44.598905 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:44.598905 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 284/553] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:44.606527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:44.646150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:44.646761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:44.647043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:44.647320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:44.648498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:44.648804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:44.650342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:44.671860 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 270/554] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 30 09:22:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:44.677444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:44.690056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:44.811577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f161a-6e7f-47e2-ad32-1e792c00f56d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:44.845378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:44.883186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:44.883797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:44.884077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:44.884353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:44.884995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:44.885299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:44.886705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:44.893127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66b3ac-c6ab-4589-9492-3b6ef379592e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:44.914063 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 285/555] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:44.919607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:44.967709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14b040-ebca-48ef-8e7f-da22249d51d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:44.968340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14b040-ebca-48ef-8e7f-da22249d51d2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:44.968621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14b040-ebca-48ef-8e7f-da22249d51d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:44.969878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14b040-ebca-48ef-8e7f-da22249d51d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.077524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14b040-ebca-48ef-8e7f-da22249d51d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:45.078959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e14b040-ebca-48ef-8e7f-da22249d51d2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:45.121044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ddb3-4929-4adb-97a3-ab87d352e016 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:45.133988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:45.162730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f15ed-a0cb-42d6-808d-c269b425d68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:45.163364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f15ed-a0cb-42d6-808d-c269b425d68c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.163911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f15ed-a0cb-42d6-808d-c269b425d68c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.164187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f15ed-a0cb-42d6-808d-c269b425d68c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.172329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f15ed-a0cb-42d6-808d-c269b425d68c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:45.174160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:45.174792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f15ed-a0cb-42d6-808d-c269b425d68c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-603f15ed-a0cb-42d6-808d-c269b425d68c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:45.189195 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:45.190619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:45.191620 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 286/556] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:22:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:45.192191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1470f-c3e6-4d22-8046-25435153193e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.192191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1470f-c3e6-4d22-8046-25435153193e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.192191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1470f-c3e6-4d22-8046-25435153193e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.193822 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 287/557] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:45.197762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df7ca-9fac-48e5-add0-4fdd8aec3152 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.197762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df7ca-9fac-48e5-add0-4fdd8aec3152 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.197762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df7ca-9fac-48e5-add0-4fdd8aec3152 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.197762 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 288/558] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:45.198085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df7ca-9fac-48e5-add0-4fdd8aec3152 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:45.205115 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 271/559] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:45.207114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7242-455a-489a-a263-14f997ceaaec None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.207451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7242-455a-489a-a263-14f997ceaaec None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.207801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7242-455a-489a-a263-14f997ceaaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.225716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-575c7242-455a-489a-a263-14f997ceaaec None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:45.226900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-575c7242-455a-489a-a263-14f997ceaaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:45.295363 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 272/560] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:45.306991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b898f90-956c-4f43-a8ff-0dd6cbbecd12 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.307280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b898f90-956c-4f43-a8ff-0dd6cbbecd12 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.307565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b898f90-956c-4f43-a8ff-0dd6cbbecd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.324842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df7ca-9fac-48e5-add0-4fdd8aec3152 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:45.334361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3b898f90-956c-4f43-a8ff-0dd6cbbecd12 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:45.335486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3b898f90-956c-4f43-a8ff-0dd6cbbecd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:45.365704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:45.366650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.367213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.367809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.368734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:45.369291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:45.371223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:45.407407 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 289/561] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:22:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:45.408660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:45.419147 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 273/562] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:45.419147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b898f90-956c-4f43-a8ff-0dd6cbbecd12 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:45.438023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:45.462993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:45.594253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b898f90-956c-4f43-a8ff-0dd6cbbecd12 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:45.600058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631cf2-3f88-48ac-8dc9-8a6e6e2d53b5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:45.637415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:45.637708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:45.638450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.638731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.638834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.639153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.639446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.639883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.650621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:45.652046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:45.652811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:45.653740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:45.670425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:45.671468 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:45.672353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:45.672353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:45.672353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:45.672518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:45.673499 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:45.674399 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:45.674399 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:45.674399 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:45.674399 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 274/563] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:45.674399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:45.681239 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 290/564] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:45.682337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:45.758585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:45.795074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:45.798289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29750c40-6fdf-45ac-bdd5-497edd5a9bb9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:45.807230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c226-3bed-490e-b5a7-7c31f42d1407 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:45.838762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:45.839379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.839656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.839928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.840585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:45.840874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:45.842300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:45.849398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b18431-4d37-4607-8e27-ad9991ec4f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:45.850332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b18431-4d37-4607-8e27-ad9991ec4f9d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:45.850819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b18431-4d37-4607-8e27-ad9991ec4f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:45.851271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b18431-4d37-4607-8e27-ad9991ec4f9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:45.855104 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 291/565] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:45.855917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:45.866347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b18431-4d37-4607-8e27-ad9991ec4f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d9b18431-4d37-4607-8e27-ad9991ec4f9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:45.871110 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:45.872012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:45.872839 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 275/566] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:45.873507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b18431-4d37-4607-8e27-ad9991ec4f9d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:45.966659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bde119-bf8c-4552-a2dc-c1f02043dfa4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:45.977467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b18431-4d37-4607-8e27-ad9991ec4f9d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:45.991328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.000261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.000868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.001135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.001425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.002368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:46.002673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.004257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.011234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2339ca9-5827-45ef-bce3-dcb27f95a63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.011859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339ca9-5827-45ef-bce3-dcb27f95a63a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.012141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339ca9-5827-45ef-bce3-dcb27f95a63a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.012417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2339ca9-5827-45ef-bce3-dcb27f95a63a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.015374 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 292/567] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:46.016163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.024090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2339ca9-5827-45ef-bce3-dcb27f95a63a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a2339ca9-5827-45ef-bce3-dcb27f95a63a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:46.029131 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:46.030082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 276/568] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:46.031465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2339ca9-5827-45ef-bce3-dcb27f95a63a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.043984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.119316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d13fb-4269-4c5a-a37d-f13adfb9f055 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.131399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2339ca9-5827-45ef-bce3-dcb27f95a63a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.152745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.153818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.154169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.154449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.162259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.163718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.164540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d31dd-d3d4-4198-bb1a-f5a95d0ca67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.165133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d31dd-d3d4-4198-bb1a-f5a95d0ca67c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.165412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d31dd-d3d4-4198-bb1a-f5a95d0ca67c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.165705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d31dd-d3d4-4198-bb1a-f5a95d0ca67c None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.166380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d31dd-d3d4-4198-bb1a-f5a95d0ca67c None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:46.166656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d31dd-d3d4-4198-bb1a-f5a95d0ca67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.168015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d31dd-d3d4-4198-bb1a-f5a95d0ca67c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:46.174822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:46.175939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:46.176822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:46.176822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:46.176822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:46.176822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:46.176822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:46.176822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:46.176822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:46.176822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:46.177252 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 277/569] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:46.177330 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 293/570] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:46.177399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b97395-ed53-4011-8680-d58ed91cca98 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.177467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.177542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b97395-ed53-4011-8680-d58ed91cca98 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.177611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b97395-ed53-4011-8680-d58ed91cca98 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.178774 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 278/571] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:46.179451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b97395-ed53-4011-8680-d58ed91cca98 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.230309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.251422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.284006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b97395-ed53-4011-8680-d58ed91cca98 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.284271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60c20c-19eb-4bb8-b1a4-9377936872c8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.318343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3110c-b79e-4f20-9a6f-5879f6112a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.318736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.318964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3110c-b79e-4f20-9a6f-5879f6112a5f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.319238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3110c-b79e-4f20-9a6f-5879f6112a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.319340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.319505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3110c-b79e-4f20-9a6f-5879f6112a5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.319606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.319853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.320510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:46.320813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.322226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.331209 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 279/572] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:46.331848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3110c-b79e-4f20-9a6f-5879f6112a5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.332655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.333457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3110c-b79e-4f20-9a6f-5879f6112a5f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-42d3110c-b79e-4f20-9a6f-5879f6112a5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:46.345038 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:46.346182 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:46.347622 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 294/573] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:46.348296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5bf94-56c0-4c97-b2a1-ffbb0bb84a46 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.348296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5bf94-56c0-4c97-b2a1-ffbb0bb84a46 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.348439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5bf94-56c0-4c97-b2a1-ffbb0bb84a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.366493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4ce5bf94-56c0-4c97-b2a1-ffbb0bb84a46 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:46.367676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4ce5bf94-56c0-4c97-b2a1-ffbb0bb84a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:46.434038 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 295/574] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:46.434886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5bf94-56c0-4c97-b2a1-ffbb0bb84a46 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.449458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41970b-a7ff-4886-929b-c88550beb337 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.469122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.490658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.491255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.491532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.491801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.492484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:46.492788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.494276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.495897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.512642 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 280/575] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:46.512642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.546182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5bf94-56c0-4c97-b2a1-ffbb0bb84a46 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.580818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.581456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.581875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.582139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.592990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.594452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:46.609069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:46.610225 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:46.611106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:46.611106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:46.611106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:46.611106 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 296/576] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:46.611106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.621597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc81bb-9978-47e2-a71f-045857bf41ed None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.661235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.661823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.662104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.662376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.672970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.674538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:46.692723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:46.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:46.694841 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:46.694841 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:46.694841 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:46.694841 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 281/577] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:46.695322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.711912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.725253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06985f47-00d5-4256-80d5-3a5d8200623a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.742061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.759797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.760403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.760687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.760962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.761679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:46.761989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.763419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.777945 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 297/578] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:46.778744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.796222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476173e-abbd-48df-a747-f3caaaa460f5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.828445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f662-3ac4-4fa7-9e53-5bd8b8ba402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.829088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f662-3ac4-4fa7-9e53-5bd8b8ba402a None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.829369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f662-3ac4-4fa7-9e53-5bd8b8ba402a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.829660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f662-3ac4-4fa7-9e53-5bd8b8ba402a None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/aa055ee122ce4e92b4da7f8222521a9e/roles/170f0bd5d6914cb8ad93877cd5cd07bb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.844906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10f662-3ac4-4fa7-9e53-5bd8b8ba402a None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=aa055ee122ce4e92b4da7f8222521a9e, role_id=170f0bd5d6914cb8ad93877cd5cd07bb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.846385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10f662-3ac4-4fa7-9e53-5bd8b8ba402a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.859462 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 282/579] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:22:46 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/aa055ee122ce4e92b4da7f8222521a9e/roles/170f0bd5d6914cb8ad93877cd5cd07bb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:46.860441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f662-3ac4-4fa7-9e53-5bd8b8ba402a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.880573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d71d9-3d70-4db2-8132-7f74dd33dc98 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:46.920593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:46.921136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:46.921418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:46.921723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:46.922395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:46.922698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:46.924126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:46.936696 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 298/580] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:46.938224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:46.954763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.971504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:46.979859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f662-3ac4-4fa7-9e53-5bd8b8ba402a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.019350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.019968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.020251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.020527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.021186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:47.021492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.022955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.040784 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 283/581] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:47.042297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.064297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f69a4e-26a8-40e8-ab1b-0091def8a18f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.107197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40dcd-07a2-435b-a6ec-a71bcb47d76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.107844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40dcd-07a2-435b-a6ec-a71bcb47d76e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.108125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40dcd-07a2-435b-a6ec-a71bcb47d76e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.108400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40dcd-07a2-435b-a6ec-a71bcb47d76e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.120583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40dcd-07a2-435b-a6ec-a71bcb47d76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-63d40dcd-07a2-435b-a6ec-a71bcb47d76e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:47.126397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:47.127501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 299/582] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:47.129112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40dcd-07a2-435b-a6ec-a71bcb47d76e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.155201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2c94b-6ef1-4b9a-8b44-baea2afdb540 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.192965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.193475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.193792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.194065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.194793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=60761ed7c27f4018a6b4791b3b060bb0) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:47.195159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.196659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.200951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:47.203176 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 284/583] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/role_assignments?user.id=60761ed7c27f4018a6b4791b3b060bb0&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:47.203999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.230204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:47.259118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40dcd-07a2-435b-a6ec-a71bcb47d76e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.307684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.308625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.309118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.309550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.321657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.323429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:47.339546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:47.341177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 30 09:22:47.342371 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:47.342917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7063672-0662-4b28-879f-829a48a58039 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.344804 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 300/584] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:47.345905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.388387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.389517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.389856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.390139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.402456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.403776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:47.420367 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:47.421342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 09:22:47.422298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:47.426095 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 285/585] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:47.427184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.483452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11c3c5-2ac7-4f3b-a4df-703c9e7d323e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.510971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:47.520745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.521365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.521654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.521952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.522791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:47.523092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.525090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.538535 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 301/586] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:47.538535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.555396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:47.586037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a7a1d-e84d-4c80-8269-2e8c36370811 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.625475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.626109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.626387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.626657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.628021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:47.628021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.629860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.652731 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 286/587] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:47.653738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.665758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be3b05-4ffa-4b7a-abd7-1017d71e6d36 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.702199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.702673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.702951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.703282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.704014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:47.704318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.705771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.730098 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 302/588] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:47.731076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.774296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bbcec-e503-45e0-baa6-938b9f56858c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.801552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:47.820803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.821547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.821865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.822136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.829531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:47.833457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.835179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:47.853716 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:47.854689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:47.855592 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:47.855592 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:47.855592 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:47.855592 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 287/589] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:47.855842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:47.877668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ff6e0-8114-4eec-aa48-232b42ad7cd3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.931714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:47.932828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:47.933335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:47.933835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:47.948701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:47.951142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:47.962750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5d08-30b4-4dc3-84ee-71c64f0449ea None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:47.972325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:47.973278 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:47.974200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:47.974200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:47.974200 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:47.977693 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 303/590] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:47.978737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.005564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.006233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.006495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.006824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.017892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.019944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:48.035548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:48.036518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:48.037407 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:48.037407 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:48.037407 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:48.041087 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 288/591] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:48.041948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.087448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:48.121883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:48.126349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8ae02-3e4e-4a34-a263-55944c1da210 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.154252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c828cb-c7fb-4e04-99a3-f5f2b70b1797 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.179231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.180529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.181305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.182231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.183361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:48.184121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.186435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.189661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.190275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.190556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.190893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.191535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:48.191842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.193218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.210654 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 304/593] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:48.212040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.212151 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 289/593] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:48.213780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.323961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14b6c4-1458-441c-b59a-a23226c52ae1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.351583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f1fc7-d2bb-4cc7-a18f-67f77130c607 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.353951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:48.359671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.360401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.360680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.360949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.361656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:48.361981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.363896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.381269 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 305/594] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:48.382508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.405593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:48.406950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.408664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.409179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.410685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.422692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.424590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:48.440398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:48.442393 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:48.443298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:48.443298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:48.443298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:48.443298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:48.443298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:48.443298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:48.443298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:48.443298 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:48.449690 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 290/595] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:48.449690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.504193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ee06f-73a8-4470-b1cb-58582a213e18 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.540065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.540716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.541255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.541367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.549864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.551593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:48.563885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:48.564822 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 09:22:48.566319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:48.568504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a04a35-d2b0-432c-93c7-44c02950e35a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.570399 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 306/596] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:48.571269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.606071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.607031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.607543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.608026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.615956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.617715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:48.630675 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:48.631642 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:48.632518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:48.632518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:48.632518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:48.632518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:48.632518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:48.632518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:48.632518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:48.632518 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:48.634882 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 291/597] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:48.635884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.661376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:48.665318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:48.683204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8b3d6-4636-4a5c-8861-f6f673310b23 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.721282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.722446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.722446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.723122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.723780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:48.724136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.725583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.745612 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 307/598] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:48.746598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.754436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4efb4e5-31b9-4a50-a440-44a2ab227256 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.792135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.793084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.793692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.794167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.795112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:48.795618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.797483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.823249 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 292/599] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:48.824427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.853880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afa28-4fbf-4ea4-9b20-77c95564e2fa None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.890338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.890949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.891288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.891828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:48.892371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:48.892678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:48.895094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:48.904689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:48.908962 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 308/600] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:48.910483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:48.919776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:48.943457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712c9f2-61c9-47c4-a221-87462eb03d00 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:48.985170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074f3c6-a10b-45d1-ac8f-459205c14edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:48.987137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074f3c6-a10b-45d1-ac8f-459205c14edb None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:48.988526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074f3c6-a10b-45d1-ac8f-459205c14edb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:48.989339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074f3c6-a10b-45d1-ac8f-459205c14edb None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/7fe7a090bd7941bb827d301aa4949857/roles/170f0bd5d6914cb8ad93877cd5cd07bb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.011030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074f3c6-a10b-45d1-ac8f-459205c14edb None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=7fe7a090bd7941bb827d301aa4949857, role_id=170f0bd5d6914cb8ad93877cd5cd07bb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:49.013033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074f3c6-a10b-45d1-ac8f-459205c14edb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:49.029706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bd0a8-23bb-4252-863b-82cae1971db4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:49.031332 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 293/601] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:22:48 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/7fe7a090bd7941bb827d301aa4949857/roles/170f0bd5d6914cb8ad93877cd5cd07bb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:49.032366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074f3c6-a10b-45d1-ac8f-459205c14edb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:49.125348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc151c-a980-46da-9790-d2adfde1d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:49.126037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc151c-a980-46da-9790-d2adfde1d696 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.126320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc151c-a980-46da-9790-d2adfde1d696 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.126594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc151c-a980-46da-9790-d2adfde1d696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.139161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc151c-a980-46da-9790-d2adfde1d696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:49.140720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc151c-a980-46da-9790-d2adfde1d696 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-50cc151c-a980-46da-9790-d2adfde1d696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:49.158706 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:49.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:49.161277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:49.161277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:49.161277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:49.161277 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 309/602] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:49.162406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951b16a-984a-45f7-87e3-14e398c4be29 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.163023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951b16a-984a-45f7-87e3-14e398c4be29 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.163303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951b16a-984a-45f7-87e3-14e398c4be29 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.169253 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 310/603] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:49.170449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951b16a-984a-45f7-87e3-14e398c4be29 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:49.189849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074f3c6-a10b-45d1-ac8f-459205c14edb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:49.231629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:49.241684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21000211-b8eb-49ea-ae11-d14d5d273e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:49.241969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21000211-b8eb-49ea-ae11-d14d5d273e6f None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.242469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21000211-b8eb-49ea-ae11-d14d5d273e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.242933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21000211-b8eb-49ea-ae11-d14d5d273e6f None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/f41e236339784d169258f06f1090b7f6/roles/170f0bd5d6914cb8ad93877cd5cd07bb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.243238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:49.262773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21000211-b8eb-49ea-ae11-d14d5d273e6f None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=f41e236339784d169258f06f1090b7f6, role_id=170f0bd5d6914cb8ad93877cd5cd07bb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:49.264335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21000211-b8eb-49ea-ae11-d14d5d273e6f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:49.280828 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 294/604] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:22:49 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/f41e236339784d169258f06f1090b7f6/roles/170f0bd5d6914cb8ad93877cd5cd07bb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:49.282434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b2ba-4b69-4da7-8638-c70bce78db9e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.282619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b2ba-4b69-4da7-8638-c70bce78db9e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.282980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b2ba-4b69-4da7-8638-c70bce78db9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.302579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cbe5b2ba-4b69-4da7-8638-c70bce78db9e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:49.303910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cbe5b2ba-4b69-4da7-8638-c70bce78db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:49.311336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951b16a-984a-45f7-87e3-14e398c4be29 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:49.382586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:49.384144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.384411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.384685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.385707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:49.386013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:49.393260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:49.418651 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 311/605] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:49.424015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:49.429363 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 295/606] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:49.517972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37f3c4-47f3-4691-8e17-dcaa88471670 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.518270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37f3c4-47f3-4691-8e17-dcaa88471670 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.518527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37f3c4-47f3-4691-8e17-dcaa88471670 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.520622 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 296/607] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:22:49.528823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806a131-15c5-4f6b-82c1-e1dfe32ce981 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.529104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806a131-15c5-4f6b-82c1-e1dfe32ce981 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.529374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806a131-15c5-4f6b-82c1-e1dfe32ce981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.554020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1806a131-15c5-4f6b-82c1-e1dfe32ce981 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:49.555662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1806a131-15c5-4f6b-82c1-e1dfe32ce981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:49.564641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa46aaa-1637-48a8-a0a9-7b87ab4b092b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:49.579153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:49.620215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:49.620865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.621144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.621415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.633680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:49.635012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:49.637483 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 297/608] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:49.662723 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:49.663818 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:49.664919 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 312/609] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:49.669699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806a131-15c5-4f6b-82c1-e1dfe32ce981 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:49.734940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:49.810416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806a131-15c5-4f6b-82c1-e1dfe32ce981 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:49.814935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:49.864255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:49.865020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:49.865392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:49.865797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:49.866669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:49.867141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:49.868682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:49.882395 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 298/610] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:49.891390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:49.944259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd718d1-5a17-42fd-a8f8-ab5c9adc0661 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:50.011441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8ddc7-2a4e-41cb-95c8-75be9fc5e708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:50.012077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8ddc7-2a4e-41cb-95c8-75be9fc5e708 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:50.012354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8ddc7-2a4e-41cb-95c8-75be9fc5e708 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:50.012626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8ddc7-2a4e-41cb-95c8-75be9fc5e708 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:50.025401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8ddc7-2a4e-41cb-95c8-75be9fc5e708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-73e8ddc7-2a4e-41cb-95c8-75be9fc5e708 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:50.030941 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:50.032013 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:50.033012 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 313/611] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:50.039442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8ddc7-2a4e-41cb-95c8-75be9fc5e708 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:50.074040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:50.080124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:50.084139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceb394-ee93-4092-9c77-bdda7449ca6e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:50.133132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af78cb2-e472-4a5f-8467-6c811abc84e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:50.134107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af78cb2-e472-4a5f-8467-6c811abc84e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:50.135768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af78cb2-e472-4a5f-8467-6c811abc84e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:50.136045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af78cb2-e472-4a5f-8467-6c811abc84e3 None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/2c1784d615d8439a9a4cb9fea305a41b/roles/170f0bd5d6914cb8ad93877cd5cd07bb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:50.154811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af78cb2-e472-4a5f-8467-6c811abc84e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=2c1784d615d8439a9a4cb9fea305a41b, role_id=170f0bd5d6914cb8ad93877cd5cd07bb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:50.156269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af78cb2-e472-4a5f-8467-6c811abc84e3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:50.164492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8ddc7-2a4e-41cb-95c8-75be9fc5e708 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:50.181507 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 299/612] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:22:49 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/2c1784d615d8439a9a4cb9fea305a41b/roles/170f0bd5d6914cb8ad93877cd5cd07bb => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:50.181507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af78cb2-e472-4a5f-8467-6c811abc84e3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:50.239166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:50.239743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:50.240021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:50.240294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:50.242123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:50.242426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:50.246862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:50.271424 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 314/613] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:50.279237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:50.410679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af78cb2-e472-4a5f-8467-6c811abc84e3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:50.455726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaff3f-d015-4082-aade-bc34f7f11354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:50.455947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaff3f-d015-4082-aade-bc34f7f11354 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:50.456093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaff3f-d015-4082-aade-bc34f7f11354 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:50.457033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaff3f-d015-4082-aade-bc34f7f11354 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:50.463352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:50.473569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaff3f-d015-4082-aade-bc34f7f11354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:50.476086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c286ddc-5201-46d0-9447-53fcd743b4eb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7ceaff3f-d015-4082-aade-bc34f7f11354 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:50.481623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:50.482748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:50.483759 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 300/614] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:50.492245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaff3f-d015-4082-aade-bc34f7f11354 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:50.505535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:50.522254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:50.523402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:50.523825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:50.524518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:50.540152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:50.541868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:50.561191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:50.562379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:50.563456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:50.563456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:50.563456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:50.565946 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 315/615] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:50.572320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:50.655947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaff3f-d015-4082-aade-bc34f7f11354 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:50.727821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:50.728850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:50.729335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:50.729915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:50.730800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:50.731322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:50.732920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:50.750046 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 301/616] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:50.758962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:50.789945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8ac5c-c2c1-4d98-a25b-028a356a3428 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:50.856241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:50.857290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:50.857799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:50.858334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:50.859177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:50.859676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:50.861268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:50.876227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:50.887791 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 316/617] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:50.893378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:50.903130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:50.912286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d239e72-c9f3-4d52-8ef4-6d1a15e138a7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:50.959226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:50.959919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:50.960196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:50.960509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:50.982849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:50.984515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:51.010084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529f83d-dd61-4dc0-ace5-f4cbd96c9fc3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:51.013740 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:51.014660 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:51.015553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:51.015553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:51.015553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:51.017746 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 302/618] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:51.029467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:51.070217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:51.071907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.072606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.075670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.087417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:51.090054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:51.118837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:51.119991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:22:51.120922 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:51.127532 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 317/619] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:22:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:51.134700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:51.244708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:51.280031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993423c9-36b3-4a7f-a273-252b317021a5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:51.288493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd0a6e-900f-46c1-88b0-f9916feb1993 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:51.333994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:51.334322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:51.335143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.335143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.335439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.336220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:51.336531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:51.340577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:51.359778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:51.360385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.360663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.360929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.361570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:51.361892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:51.367748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:51.380150 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 318/620] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:22:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:51.385832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:51.394826 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 303/621] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:51.400564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:51.556990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253b4fe-1db7-4f56-959c-3a64f8e9b13b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:51.585935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:51.600033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf457a3-cffb-48a3-b610-2a0389479e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:51.600658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf457a3-cffb-48a3-b610-2a0389479e84 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.600998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf457a3-cffb-48a3-b610-2a0389479e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.601276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf457a3-cffb-48a3-b610-2a0389479e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.612104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec850ebe-e216-47c4-9203-308e936f2ca8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:51.679454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf457a3-cffb-48a3-b610-2a0389479e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:51.680912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf457a3-cffb-48a3-b610-2a0389479e84 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:51.684284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce45a43-6578-41ca-9749-888171bb5bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:51.684917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45a43-6578-41ca-9749-888171bb5bdd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.685193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45a43-6578-41ca-9749-888171bb5bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.685461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce45a43-6578-41ca-9749-888171bb5bdd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.693880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce45a43-6578-41ca-9749-888171bb5bdd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:51.695340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce45a43-6578-41ca-9749-888171bb5bdd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2ce45a43-6578-41ca-9749-888171bb5bdd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:51.708905 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:51.709904 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:51.710849 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 304/622] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:22:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:51.712972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3014a6-9358-4850-aa85-f5f1dc97a760 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.712972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3014a6-9358-4850-aa85-f5f1dc97a760 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.712972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3014a6-9358-4850-aa85-f5f1dc97a760 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.716784 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 305/623] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:51.718187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb76e7-eb31-4d42-927b-9f8cebf84fce None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.718479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb76e7-eb31-4d42-927b-9f8cebf84fce None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.718754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb76e7-eb31-4d42-927b-9f8cebf84fce None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.720116 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 306/624] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:51.720987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14101c-fefe-43d8-bb9a-1e85a8577a12 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.721265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14101c-fefe-43d8-bb9a-1e85a8577a12 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.721534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14101c-fefe-43d8-bb9a-1e85a8577a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.741264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5e14101c-fefe-43d8-bb9a-1e85a8577a12 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:51.742552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5e14101c-fefe-43d8-bb9a-1e85a8577a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:51.783743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:51.789496 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 319/625] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 09:22:51.790166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf457a3-cffb-48a3-b610-2a0389479e84 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:51.817144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:51.829665 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 307/626] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:51.829665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd2754-e97d-4b38-b9ef-55388088f519 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.829665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd2754-e97d-4b38-b9ef-55388088f519 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.829665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd2754-e97d-4b38-b9ef-55388088f519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.845385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f9cd2754-e97d-4b38-b9ef-55388088f519 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:51.846670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f9cd2754-e97d-4b38-b9ef-55388088f519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:51.908267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf457a3-cffb-48a3-b610-2a0389479e84 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:51.921574 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 308/627] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:51.922536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd2754-e97d-4b38-b9ef-55388088f519 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:51.983703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:51.984499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:51.989046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:51.989046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:51.989046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:51.989046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:51.989046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.017573 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 320/628] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 30 09:22:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:52.025874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.072042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd2754-e97d-4b38-b9ef-55388088f519 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.074994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:52.127978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.129032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.129306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.130462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.143424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.145871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.159256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:52.163331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:52.164661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:52.165846 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:52.165846 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:52.165846 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:52.167484 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 309/629] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:52.168448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.184364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22f237-2fa1-440d-8bac-299fd55f44e7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.224265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.225145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.225605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.226070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.238331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.240158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:52.258878 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:52.260033 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:52.261107 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:52.261107 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:52.261107 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:52.262563 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 321/630] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:52.267482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.294012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee2bdc-eef7-4f22-9934-35dd1ade1c75 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.327321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633f7e6-d8f0-462b-8d1a-8d87a01e6b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.327872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633f7e6-d8f0-462b-8d1a-8d87a01e6b05 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.328150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633f7e6-d8f0-462b-8d1a-8d87a01e6b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.328421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633f7e6-d8f0-462b-8d1a-8d87a01e6b05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.338933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633f7e6-d8f0-462b-8d1a-8d87a01e6b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f633f7e6-d8f0-462b-8d1a-8d87a01e6b05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:52.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:52.345234 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:52.346275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:52.350631 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 310/631] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:52.351336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633f7e6-d8f0-462b-8d1a-8d87a01e6b05 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.371110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:52.380138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f544a-a587-4db4-9b59-e9cf80610d0f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.426538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.427232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.427578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.427946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.428826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:52.429240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.430940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.432758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:52.446030 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 322/632] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:52.447063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.461320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633f7e6-d8f0-462b-8d1a-8d87a01e6b05 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.498214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f78cc-6987-4515-ac7a-8c8a40992cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.498842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f78cc-6987-4515-ac7a-8c8a40992cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.499124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f78cc-6987-4515-ac7a-8c8a40992cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.499394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f78cc-6987-4515-ac7a-8c8a40992cc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.511772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f78cc-6987-4515-ac7a-8c8a40992cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-586f78cc-6987-4515-ac7a-8c8a40992cc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:52.516970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:52.518096 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:52.519966 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 311/633] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:52.520697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f78cc-6987-4515-ac7a-8c8a40992cc4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.563841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ea71e-ebf7-4662-9ddb-63fc54b27963 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.600322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.600747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:52.600857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.601118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.601385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.602058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:52.602361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.603724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.615533 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 323/634] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:52.616365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.629549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f78cc-6987-4515-ac7a-8c8a40992cc4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.664422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.665024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.665300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.665567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.666250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:52.666546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.667912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.674725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:52.676317 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 312/635] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:52.676824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.728011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4f526-837f-4fef-b324-8f415f2ced04 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.763971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.764619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.764918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.765180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.773447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.774970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.775308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3e7fe-a7ef-4f86-9f91-3bf1b41f494b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:52.791167 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:52.792168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:52.793298 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 324/636] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:52.794044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.821085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.821732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.822015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.822292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.822951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:52.823255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.824629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.830212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:52.835767 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 313/637] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:52.836929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.906488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba5611-2b40-467d-827a-a41c93341ea0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.920240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:52.940336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.940968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd580ee4-6a3d-46cb-b574-d0cf292fe795 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:52.941111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.941295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.941521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.948026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.949469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:52.960482 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:52.961391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:52.962345 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 325/638] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:52.962819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:52.977295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:52.977884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:52.978169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:52.978444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:52.989456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:52.990938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:53.006906 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:53.008111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:53.009062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:53.009062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:53.009062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:53.009062 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 314/639] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:53.009062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.062061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:53.065075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e5ad9-2311-49f6-86e8-d8464633bb18 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.098919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a99d32-b3f8-4f6b-8488-86b7391649db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.099526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99d32-b3f8-4f6b-8488-86b7391649db None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.099808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99d32-b3f8-4f6b-8488-86b7391649db None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.100060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99d32-b3f8-4f6b-8488-86b7391649db None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.100736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a99d32-b3f8-4f6b-8488-86b7391649db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:53.101034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a99d32-b3f8-4f6b-8488-86b7391649db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.102474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a99d32-b3f8-4f6b-8488-86b7391649db None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.114874 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 326/640] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:53.115365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454154e-d3b2-40cb-a3a3-232e21fe358c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.116325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406a51-742f-4de7-9875-047f7fb34a2c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.116708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406a51-742f-4de7-9875-047f7fb34a2c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.117093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83406a51-742f-4de7-9875-047f7fb34a2c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.121663 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 327/641] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:53.121663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83406a51-742f-4de7-9875-047f7fb34a2c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.156439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.156727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:53.157061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.157430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.157614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.168177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.169666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:53.183525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:53.184663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:53.185655 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:53.185655 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:53.185655 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:53.185655 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 315/642] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:53.185655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.224310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83406a51-742f-4de7-9875-047f7fb34a2c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.258620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.259249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.259526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.259793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.260480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:53.260786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.262293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.274454 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 328/643] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:53.275348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.286002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d435ab-92ed-4a7a-b2f9-a2e7899f75e9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.288357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:53.321215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd573a56-248e-4b88-9eea-051356d97e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.321842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573a56-248e-4b88-9eea-051356d97e43 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.322118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573a56-248e-4b88-9eea-051356d97e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.322386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd573a56-248e-4b88-9eea-051356d97e43 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.323021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd573a56-248e-4b88-9eea-051356d97e43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:53.323317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd573a56-248e-4b88-9eea-051356d97e43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.324680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd573a56-248e-4b88-9eea-051356d97e43 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.336251 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 316/644] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:53.337219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef7c3-3b1c-4e6d-a5a9-902ae8867d29 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.337503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef7c3-3b1c-4e6d-a5a9-902ae8867d29 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.337807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fef7c3-3b1c-4e6d-a5a9-902ae8867d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.354719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a6fef7c3-3b1c-4e6d-a5a9-902ae8867d29 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:53.355896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a6fef7c3-3b1c-4e6d-a5a9-902ae8867d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:53.371250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:53.378007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b174f-ba80-4fde-b73a-0c1db1d78ebb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.416344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.416901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.417179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.418027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.418765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=aa055ee122ce4e92b4da7f8222521a9e) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:53.419071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.420496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.426083 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 329/645] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/role_assignments?user.id=aa055ee122ce4e92b4da7f8222521a9e&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:53.437392 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 317/646] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:53.437599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.437741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fef7c3-3b1c-4e6d-a5a9-902ae8867d29 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.557109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:53.562333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cffd1f-1bf1-4ae7-99d0-1c7724c1fc29 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.567482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fef7c3-3b1c-4e6d-a5a9-902ae8867d29 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.603812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.604431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.606278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.606278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.606278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:53.606278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.607629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.620203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15794ee2-f47f-44d5-bb82-0b7470ff621b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.621187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15794ee2-f47f-44d5-bb82-0b7470ff621b None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.623364 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 330/647] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:53.623732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15794ee2-f47f-44d5-bb82-0b7470ff621b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.624110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15794ee2-f47f-44d5-bb82-0b7470ff621b None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/60761ed7c27f4018a6b4791b3b060bb0/roles/170f0bd5d6914cb8ad93877cd5cd07bb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.625537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.646640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15794ee2-f47f-44d5-bb82-0b7470ff621b None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=60761ed7c27f4018a6b4791b3b060bb0, role_id=170f0bd5d6914cb8ad93877cd5cd07bb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.648124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15794ee2-f47f-44d5-bb82-0b7470ff621b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.652811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:53.666634 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 318/648] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:22:53 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/60761ed7c27f4018a6b4791b3b060bb0/roles/170f0bd5d6914cb8ad93877cd5cd07bb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:22:53.667552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15794ee2-f47f-44d5-bb82-0b7470ff621b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.766427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b1406-a4fd-413c-8eb9-61adeb3a1b97 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.802943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15794ee2-f47f-44d5-bb82-0b7470ff621b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:53.822853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.823517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.823802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.824078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.832605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.834141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.843456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:53.844153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:53.844438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:53.844713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] PATH_INFO: `/v3/users/glance` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:53.848916 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:53.849933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:53.851319 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 331/649] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:53.852251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.853730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:53.855292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:53.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:53.871000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:53.871917 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 319/650] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:53.872489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:53.878208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:53.954307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:53.989649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47ab19-35b5-4ff4-9b89-4aa3af830a60 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:54.007758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bd75b-a93c-45b0-9b26-034f28f758ca None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:54.040164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdd6d6-93cf-4636-845e-1cd43a3b0c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:54.041856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd6d6-93cf-4636-845e-1cd43a3b0c0b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:54.042345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd6d6-93cf-4636-845e-1cd43a3b0c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:54.042820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd6d6-93cf-4636-845e-1cd43a3b0c0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:54.045514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:54.046139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:54.046645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:54.046645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:54.047486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:54.047790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:54.049230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:54.060608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdd6d6-93cf-4636-845e-1cd43a3b0c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-bfbdd6d6-93cf-4636-845e-1cd43a3b0c0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:54.068348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:54.069335 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:54.070212 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 332/651] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:54.072188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdd6d6-93cf-4636-845e-1cd43a3b0c0b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:54.082325 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 320/652] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 30 09:22:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:54.083715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:54.154000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:54.198421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdd6d6-93cf-4636-845e-1cd43a3b0c0b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:54.203410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e436514-d0a0-488d-a0fc-a8905d179255 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:54.238381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:54.238899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:54.239174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:54.239442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:54.240162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:54.240462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:54.241908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:54.250741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:54.251406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:54.252012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:54.252494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:54.254472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:54.255361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:54.255873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:54.257527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:54.267090 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 333/653] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 30 09:22:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:54.267090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:54.273320 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 321/654] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:54.278987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:54.389377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36872-4804-47b3-91da-2d1b278d49fa None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:54.417139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f831b-c733-4986-8168-661c58cf0f16 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:54.443179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:54.447494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09bf8c-7f91-4996-815c-61b295d20c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:54.448219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09bf8c-7f91-4996-815c-61b295d20c59 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:54.448510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09bf8c-7f91-4996-815c-61b295d20c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:54.448789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09bf8c-7f91-4996-815c-61b295d20c59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:54.458193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177eaf-b02f-410f-959c-891828a6c68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:54.459143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177eaf-b02f-410f-959c-891828a6c68d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:54.459633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177eaf-b02f-410f-959c-891828a6c68d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:54.460096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177eaf-b02f-410f-959c-891828a6c68d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:54.535389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:54.535862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff177eaf-b02f-410f-959c-891828a6c68d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:54.537574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff177eaf-b02f-410f-959c-891828a6c68d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:54.544797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09bf8c-7f91-4996-815c-61b295d20c59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:54.546303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09bf8c-7f91-4996-815c-61b295d20c59 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:54.655838 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 322/655] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:54.657375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177eaf-b02f-410f-959c-891828a6c68d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:54.676223 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 334/656] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:54.678451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09bf8c-7f91-4996-815c-61b295d20c59 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:54.690702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:54.776660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177eaf-b02f-410f-959c-891828a6c68d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:54.797823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:54.808858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09bf8c-7f91-4996-815c-61b295d20c59 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:54.819528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:54.820513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:54.821020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:54.821487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:54.832698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:54.834484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:54.849360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:54.850021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:54.850302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:54.850576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:54.851447 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:54.852326 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:54.853184 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:54.853184 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:54.853184 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:54.859653 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 323/657] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:54.860730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:54.861462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:54.863987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:54.881085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:54.882307 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:54.883186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:54.883186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:54.883186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:54.885867 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 335/658] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:54.886593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:54.950440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:54.980682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c755e3-496d-4eb2-872d-f2750c645d79 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.006147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f1111-9c51-47b8-a7b0-6800af761d80 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.031219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.032790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.033498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.034033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.044806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.045941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.046323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.046699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.047883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:55.048233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.049574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.050186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.052066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.065771 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 336/659] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:55.065771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:55.074827 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:55.075957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:55.077029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:55.077029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:55.077029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:55.077029 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 324/660] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:55.077029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.083412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:55.181304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2e952-7fb4-413c-b8a1-cc80b57791c1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.209112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c650d-1e66-48e2-9069-3bbd90308a9f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.210135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:55.216161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.216789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.217073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.217349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.226285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.227850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:55.240372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:55.241325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 325/661] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:55.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.262030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.263367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.263851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.264325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.265168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:55.266204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.268085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.289910 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 337/662] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:55.289910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.364520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:55.376364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca1441-b3a2-45c3-b63a-74fa712d44ea None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.406267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42e722-e1ce-4d71-9736-0ec75fe7be7c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.416580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424df201-cbff-4792-9781-550adf567829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.417303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424df201-cbff-4792-9781-550adf567829 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.417748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424df201-cbff-4792-9781-550adf567829 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.418195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424df201-cbff-4792-9781-550adf567829 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.418940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424df201-cbff-4792-9781-550adf567829 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:55.419346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424df201-cbff-4792-9781-550adf567829 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.420941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424df201-cbff-4792-9781-550adf567829 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.436128 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 326/663] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:55.437185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424df201-cbff-4792-9781-550adf567829 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.447691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.448312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.448644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.448922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.449778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:55.450084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.451584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.470761 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 338/664] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:55.472037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.493902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:55.554786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424df201-cbff-4792-9781-550adf567829 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.577429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b80d3e-121b-4cae-a2c0-4fb417ddfe07 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.594201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.595247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.595800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.596285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.604174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:55.605464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.607330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.613403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.614157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.614499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.614892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.621989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:55.622458 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:55.623466 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:55.624559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:55.624559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:55.624559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:55.624559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:55.624559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:55.624559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:55.624559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:55.624559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:55.625937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.632031 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 327/665] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:55.633160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:55.638868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:55.639889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:55.640962 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:55.640962 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:55.640962 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:55.640962 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:55.640962 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:55.640962 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:55.640962 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:55.640962 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:55.647864 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 339/666] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:55.648923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.754228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba7cdc-abc3-4f84-9820-1c3bec4e3db6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.756892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beca97-767b-4ac0-8373-84c303658924 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.765348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:55.793129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.793968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.794353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.794732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.795527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:55.795943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.796964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c56b1-ca30-4d34-b73f-8b3e2a31274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.797659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.797943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c56b1-ca30-4d34-b73f-8b3e2a31274b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.798383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c56b1-ca30-4d34-b73f-8b3e2a31274b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.798799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c56b1-ca30-4d34-b73f-8b3e2a31274b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.817673 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 340/667] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:55.817673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.863354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:55.884477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c56b1-ca30-4d34-b73f-8b3e2a31274b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.886373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c56b1-ca30-4d34-b73f-8b3e2a31274b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.927332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf3975-ac81-4c80-9d1a-8becec1258d2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:55.963336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:55.963947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:55.964232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:55.964497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:55.965190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:55.965550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:55.967012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:55.979999 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 341/668] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:55.980655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:55.999786 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 328/669] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:56.001037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6239-b9b6-44aa-845b-0f0015f15a21 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.001327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6239-b9b6-44aa-845b-0f0015f15a21 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.001603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6239-b9b6-44aa-845b-0f0015f15a21 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.003526 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 329/670] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:56.007400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2855c82-aafb-4386-a2ad-65e2e7310bad None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.007711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2855c82-aafb-4386-a2ad-65e2e7310bad None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.008006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2855c82-aafb-4386-a2ad-65e2e7310bad None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.009511 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 330/671] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:56.010191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2855c82-aafb-4386-a2ad-65e2e7310bad None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:56.021788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:56.080432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828f89-e7c1-416b-a847-f1c9e8f2c896 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:56.088542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:56.111550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2855c82-aafb-4386-a2ad-65e2e7310bad None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:56.116646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:56.117241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.117540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.117869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.118580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=7fe7a090bd7941bb827d301aa4949857) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:56.118883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:56.120312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:56.125794 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 342/672] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:55 2022] GET /identity/v3/role_assignments?user.id=7fe7a090bd7941bb827d301aa4949857&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:56.126554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:56.165565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd8283-30d3-4511-89a4-490ac4e57474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:56.166134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd8283-30d3-4511-89a4-490ac4e57474 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.166414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd8283-30d3-4511-89a4-490ac4e57474 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.166685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd8283-30d3-4511-89a4-490ac4e57474 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.167459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd8283-30d3-4511-89a4-490ac4e57474 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=f41e236339784d169258f06f1090b7f6) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:56.167869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd8283-30d3-4511-89a4-490ac4e57474 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:56.169379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd8283-30d3-4511-89a4-490ac4e57474 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:56.177150 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 331/673] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:56 2022] GET /identity/v3/role_assignments?user.id=f41e236339784d169258f06f1090b7f6&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:56.178465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a92f3-5020-4df5-8daa-50f4f117bd08 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.178751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a92f3-5020-4df5-8daa-50f4f117bd08 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.179022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a92f3-5020-4df5-8daa-50f4f117bd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.205115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-426a92f3-5020-4df5-8daa-50f4f117bd08 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:56.208043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-426a92f3-5020-4df5-8daa-50f4f117bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:56.253944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9736c6d-516b-4fe8-8302-77a6e6a1df4a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:56.296826 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 332/674] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:56.297490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:56.298188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.298468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.298740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.299670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837894a7-62ba-42f8-9c27-51a100c9c97f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.300749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837894a7-62ba-42f8-9c27-51a100c9c97f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.301397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837894a7-62ba-42f8-9c27-51a100c9c97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.308463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:56.313687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:56.313687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:56.329018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-837894a7-62ba-42f8-9c27-51a100c9c97f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:56.330525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-837894a7-62ba-42f8-9c27-51a100c9c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:56.330942 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:56.332026 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:56.333123 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:56.333123 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:56.333123 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:56.333123 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 343/675] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:56.339477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:56.366671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:56.418907 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 333/676] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:56.449812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8663068-7ca0-4b25-8dfa-b187669531b4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:56.483755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:56.484367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.484646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.484921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.485561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:56.485893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:56.487995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:56.500146 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 344/677] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:56.506342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837894a7-62ba-42f8-9c27-51a100c9c97f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:56.617815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837894a7-62ba-42f8-9c27-51a100c9c97f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:56.632884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:56.655699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:56.668726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:56.669430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.671100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.671379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.685561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:56.687056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:56.709960 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:56.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:56.712136 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 334/678] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:56.712630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:56.750687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:56.772466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17612a-451b-46fe-a409-eda059012644 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:56.809818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddfda9-32d4-4b23-924b-89d43efebabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:56.810590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddfda9-32d4-4b23-924b-89d43efebabc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.811032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddfda9-32d4-4b23-924b-89d43efebabc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.811410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddfda9-32d4-4b23-924b-89d43efebabc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.824469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddfda9-32d4-4b23-924b-89d43efebabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b7ddfda9-32d4-4b23-924b-89d43efebabc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:56.829677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:56.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:22:56.832232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:56.838966 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 345/679] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:22:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:22:56.838966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddfda9-32d4-4b23-924b-89d43efebabc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:56.858133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63da15-a42c-4409-9fbc-046073c477e6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:56.893819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac194e-def6-454f-b67b-36e836d87063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:56.894818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac194e-def6-454f-b67b-36e836d87063 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:56.895223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac194e-def6-454f-b67b-36e836d87063 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:56.895605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac194e-def6-454f-b67b-36e836d87063 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:56.906527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac194e-def6-454f-b67b-36e836d87063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-26ac194e-def6-454f-b67b-36e836d87063 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:22:56.912607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:56.913718 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:22:56.914766 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:56.916724 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 335/680] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:22:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:56.917705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac194e-def6-454f-b67b-36e836d87063 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:56.967372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddfda9-32d4-4b23-924b-89d43efebabc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:56.982003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:57.016145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.017459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.017994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.018839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.020535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:57.021373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.024381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:57.029471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac194e-def6-454f-b67b-36e836d87063 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.041816 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 346/681] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:57.042984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:57.047515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:57.068791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.069579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.070052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.070436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.073436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:57.073436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.073436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:57.088713 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 336/682] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:22:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:57.088713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:57.184035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6954-1950-4a56-8170-72793403a400 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.199740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8430d-7197-4520-9088-7e0bcbf14d27 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.216750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:57.227271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.228095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.228681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.229108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.230576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:57.231039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.232568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:57.241552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.242320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.242697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.243070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.243858 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 347/683] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:22:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:57.244809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:57.248933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=2c1784d615d8439a9a4cb9fea305a41b) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:57.249495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.251174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:57.259082 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 337/684] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:57 2022] GET /identity/v3/role_assignments?user.id=2c1784d615d8439a9a4cb9fea305a41b&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:57.260187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:57.344279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:57.387228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681dadb-36ec-4eab-8465-a72f8b71fb43 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.394631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a804d-5f16-4cd4-b012-9904ca0750e6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.439396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.440246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.440425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.441256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.441621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.442105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.442536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.442975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.453875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.455534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:57.457023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.458946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:57.477753 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:57.478720 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:57.479627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:57.479627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:57.479627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:57.481562 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 348/685] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:57.483840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:57.487378 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:57.488923 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:57.490063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:57.490063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:57.490063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:57.498346 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 338/686] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:57.506295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:57.574912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:57.666000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bb29d-9bf8-4dfc-939a-0200b9449d25 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.673331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2cfa9-04ad-4fcd-a691-e61d0d62575e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.694557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:57.710368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.710981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.711257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.711527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.712340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:57.712643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.716168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:57.724718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.725653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.726575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.727354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.728350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:57.728970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.730620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:57.733739 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 339/687] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:57.735683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:57.747987 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 349/688] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:57.756258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:57.897991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:57.916666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742c027-3e38-4b1e-a96b-9e7fe0a70dc9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.937110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18551cb-fb34-40bf-9efa-9ddf32786002 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:57.983799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:57.984347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:57.984624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:57.984890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:57.994017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:57.996375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:58.001049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:58.001698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:58.001973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:58.002238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:58.013567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:58.020303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:58.021103 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:58.022273 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:58.023538 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 340/689] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:22:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:58.029693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:22:58.046635 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:58.049110 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:58.052427 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 350/690] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:22:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:58.059677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:58.128269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:58.220509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcf5bf-c9a9-40a5-9a84-acbbdeda4dc4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:58.238608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffd14a-fdfc-4400-b444-62a43d4ce7ac None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:58.286362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:58.287248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:58.287564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:58.287853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:58.288602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:58.288917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:58.290496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:58.292628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:58.293477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:58.293995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:58.294477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:58.295470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:58.295979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:58.297594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:58.320825 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 341/691] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 30 09:22:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:58.326751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:58.333810 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 351/692] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:22:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:22:58.343706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:58.351796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:58.522417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080c5af-9ba2-4370-9a0b-076395fb4561 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:58.547882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:58.568043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cf396-f118-42e0-8ffa-e4c9a1a67711 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:58.572837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd75de6-dd20-47d7-8831-de5e6d6a84c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:58.573475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd75de6-dd20-47d7-8831-de5e6d6a84c6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:58.573793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd75de6-dd20-47d7-8831-de5e6d6a84c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:58.574065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd75de6-dd20-47d7-8831-de5e6d6a84c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:58.651319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0f481-d3f6-4d32-b5e0-062f6d8590a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:58.651957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0f481-d3f6-4d32-b5e0-062f6d8590a0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:58.652236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0f481-d3f6-4d32-b5e0-062f6d8590a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:58.652508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0f481-d3f6-4d32-b5e0-062f6d8590a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:58.718348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd75de6-dd20-47d7-8831-de5e6d6a84c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:58.719835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd75de6-dd20-47d7-8831-de5e6d6a84c6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:58.740768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:58.781459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0f481-d3f6-4d32-b5e0-062f6d8590a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:58.782808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0f481-d3f6-4d32-b5e0-062f6d8590a0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:58.865139 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 342/693] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 30 09:22:58.866701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b8247-801a-4c57-9e5c-085ce1386245 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:58.867217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b8247-801a-4c57-9e5c-085ce1386245 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:58.867337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b8247-801a-4c57-9e5c-085ce1386245 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:58.872635 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 343/694] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:58.872635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b8247-801a-4c57-9e5c-085ce1386245 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:58.910240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:58.919902 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 352/695] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:22:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 30 09:22:58.921563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c51a2-11f9-4de5-81f9-f474b4e83292 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:58.922096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c51a2-11f9-4de5-81f9-f474b4e83292 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:58.922578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c51a2-11f9-4de5-81f9-f474b4e83292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:58.944062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-568c51a2-11f9-4de5-81f9-f474b4e83292 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:58.946901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-568c51a2-11f9-4de5-81f9-f474b4e83292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:58.988527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b8247-801a-4c57-9e5c-085ce1386245 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.006400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:59.028516 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 353/696] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:59.029835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c51a2-11f9-4de5-81f9-f474b4e83292 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.045297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fe979-fb87-46d4-97ee-82e56f7cbb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.045900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fe979-fb87-46d4-97ee-82e56f7cbb07 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.046177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fe979-fb87-46d4-97ee-82e56f7cbb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.046451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fe979-fb87-46d4-97ee-82e56f7cbb07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.058282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fe979-fb87-46d4-97ee-82e56f7cbb07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.059844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fe979-fb87-46d4-97ee-82e56f7cbb07 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-005fe979-fb87-46d4-97ee-82e56f7cbb07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:59.077596 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:59.078582 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:59.079533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:59.079533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:59.079533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:59.087192 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 344/697] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:59.089387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb350-469b-4591-acde-0d5cfb1ffbda None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.090116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb350-469b-4591-acde-0d5cfb1ffbda None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.090625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb350-469b-4591-acde-0d5cfb1ffbda None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.096250 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 345/698] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:22:59.098105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcb350-469b-4591-acde-0d5cfb1ffbda None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.140899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c51a2-11f9-4de5-81f9-f474b4e83292 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.166336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:59.179428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984562-4ce1-4c48-a8fc-4b7ef8bffb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.180062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984562-4ce1-4c48-a8fc-4b7ef8bffb1e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.180375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984562-4ce1-4c48-a8fc-4b7ef8bffb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.180596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984562-4ce1-4c48-a8fc-4b7ef8bffb1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.191162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79984562-4ce1-4c48-a8fc-4b7ef8bffb1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.192622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79984562-4ce1-4c48-a8fc-4b7ef8bffb1e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-79984562-4ce1-4c48-a8fc-4b7ef8bffb1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:59.208769 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:59.209838 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:59.210767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:59.210767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:59.210767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:59.210767 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 354/699] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:59.210994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e36e0-05ca-4cf9-991c-85c71201e424 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.212691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e36e0-05ca-4cf9-991c-85c71201e424 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.212691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e36e0-05ca-4cf9-991c-85c71201e424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.230313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3e3e36e0-05ca-4cf9-991c-85c71201e424 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:22:59.231484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e3e36e0-05ca-4cf9-991c-85c71201e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:22:59.244209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcb350-469b-4591-acde-0d5cfb1ffbda None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.293295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.294589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.295571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.296394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.297579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:59.298539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.300927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.302932 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 355/700] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:22:59.304063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e36e0-05ca-4cf9-991c-85c71201e424 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.327900 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 346/701] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:59.334773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.361828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:59.471055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14077bf-c006-4fee-99ae-5b2b4a7f423e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.476172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e36e0-05ca-4cf9-991c-85c71201e424 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.484041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:59.510141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.510958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.511349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.511837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.515129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.516049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.516643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.517263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.518261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:59.519030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.519712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.520954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.521158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:59.533019 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:59.533959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:59.535001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:59.535001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:59.535001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:59.535001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:59.535001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:59.535001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:59.535001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:59.535001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:59.541668 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 347/702] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:59.541668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.542237 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 356/703] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:59.543381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.629246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:59.651325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec4d5-d000-4b28-8601-5ae14bf46621 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.658179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d6c81-783b-471b-be47-4ab491369191 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.689214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.690029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.691175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.691462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] PATH_INFO: `/v3/services/network` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.692047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.693442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.696731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.697530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.697999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.698425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.699335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:22:59.699790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.701359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:59.704537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:59.705850 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:59.707397 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 348/704] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:59.711684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.719498 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 357/705] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:22:59.720977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.754683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:59.833713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c34b1-9ea6-49a6-8ba9-ff0b9d4dfb36 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.835207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b60e6-3e7b-480c-b6d4-d813f407457f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:22:59.872098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.872732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.873013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.873287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.874779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:22:59.875576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:22:59.876178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:22:59.876461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:22:59.880259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.881694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.887326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:22:59.888761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:22:59.892949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:59.894318 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:59.895232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:22:59.895232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:22:59.895232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:22:59.895232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:22:59.895232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:22:59.895232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:22:59.895232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:22:59.895232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:22:59.898001 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 349/706] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:59.898864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.903068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:22:59.907161 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:22:59.908133 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:22:59.910899 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:22:59.910899 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:22:59.910899 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:22:59.910899 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 358/707] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:22:59.910899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:22:59.993850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:00.014474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68644d26-ae30-402b-9013-40af21e397c7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:00.018979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee443205-5649-4597-9eca-75011e44c99a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:00.048508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:00.049112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.049395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.049807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.050349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] RBAC: Adding query filter params (name=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:00.050653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:00.052041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:00.052646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:00.053245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.053521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.053827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.054544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=60761ed7c27f4018a6b4791b3b060bb0) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:00.054852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:00.056270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:00.056452 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 350/708] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:00.057176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:00.062212 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 359/709] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:22:59 2022] GET /identity/v3/role_assignments?user.id=60761ed7c27f4018a6b4791b3b060bb0&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:00.062859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:00.146946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:00.181324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e2128-3917-42dc-b62f-00a742171077 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:00.184150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161e60-14e2-40fa-8bc5-a4542d6e9537 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:00.218148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817fb68-474b-477a-8f91-334adac8a6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:00.218766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817fb68-474b-477a-8f91-334adac8a6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.219048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817fb68-474b-477a-8f91-334adac8a6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.219321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817fb68-474b-477a-8f91-334adac8a6d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.219966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817fb68-474b-477a-8f91-334adac8a6d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:00.220267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817fb68-474b-477a-8f91-334adac8a6d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:00.222279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817fb68-474b-477a-8f91-334adac8a6d9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:00.222564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:00.223184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.223457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.223890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.224422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:00.224724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:00.226201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:00.240518 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 360/710] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:00.240518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50fea9-c3f9-4222-ae1b-8c1d000a0571 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.240518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50fea9-c3f9-4222-ae1b-8c1d000a0571 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.240518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50fea9-c3f9-4222-ae1b-8c1d000a0571 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.240518 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 361/711] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:00.240518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50fea9-c3f9-4222-ae1b-8c1d000a0571 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:00.243417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:00.247425 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 351/712] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:00.248909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:00.360961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50fea9-c3f9-4222-ae1b-8c1d000a0571 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:00.362058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d0814-60f7-459b-916d-df46111f6163 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:00.394500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:00.404524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50625d64-46c0-4217-bd5a-dfe72f90ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:00.404683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5e96c-462c-4805-90a2-887ed8a78804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:00.405091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5e96c-462c-4805-90a2-887ed8a78804 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.405195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625d64-46c0-4217-bd5a-dfe72f90ee6f None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.405383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5e96c-462c-4805-90a2-887ed8a78804 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.405549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625d64-46c0-4217-bd5a-dfe72f90ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.405751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5e96c-462c-4805-90a2-887ed8a78804 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.405922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625d64-46c0-4217-bd5a-dfe72f90ee6f None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.408841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5e96c-462c-4805-90a2-887ed8a78804 None admin] RBAC: Adding query filter params (type=network) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:00.408956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50625d64-46c0-4217-bd5a-dfe72f90ee6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:00.409531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5e96c-462c-4805-90a2-887ed8a78804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:00.409531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5e96c-462c-4805-90a2-887ed8a78804 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:00.409719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50625d64-46c0-4217-bd5a-dfe72f90ee6f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:00.415193 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 362/713] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:00.416550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a218934-f30f-4835-bfe3-4b5db788a37b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.416904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a218934-f30f-4835-bfe3-4b5db788a37b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.417252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a218934-f30f-4835-bfe3-4b5db788a37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.441845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1a218934-f30f-4835-bfe3-4b5db788a37b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:00.443413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1a218934-f30f-4835-bfe3-4b5db788a37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:00.444207 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 352/714] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:23:00 2022] POST /identity/v3/users => generated 254 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:00.445232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50625d64-46c0-4217-bd5a-dfe72f90ee6f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:00.574135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50625d64-46c0-4217-bd5a-dfe72f90ee6f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:00.615707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:00.616455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:00.616805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:00.617155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:00.618034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=aa055ee122ce4e92b4da7f8222521a9e) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:00.618413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:00.620156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:00.626941 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 353/715] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:00 2022] GET /identity/v3/role_assignments?user.id=aa055ee122ce4e92b4da7f8222521a9e&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:00.636121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:00.647071 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 363/716] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:23:00.942020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:01.021349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:01.101059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54e9e1-6a4e-4454-b8d8-64d341c70582 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:01.146714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:01.147373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:01.147752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:01.148056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:01.148664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:01.150151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:01.162925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:01.163952 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:01.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 354/717] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 30 09:23:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:01.170254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a218934-f30f-4835-bfe3-4b5db788a37b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:01.317715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a218934-f30f-4835-bfe3-4b5db788a37b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:01.362720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aad57e-bb6b-4997-a367-0e1811f0dbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:01.363432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad57e-bb6b-4997-a367-0e1811f0dbcb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:01.363718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad57e-bb6b-4997-a367-0e1811f0dbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:01.363996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad57e-bb6b-4997-a367-0e1811f0dbcb None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:01.364636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aad57e-bb6b-4997-a367-0e1811f0dbcb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:01.364944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aad57e-bb6b-4997-a367-0e1811f0dbcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:01.366349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aad57e-bb6b-4997-a367-0e1811f0dbcb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:01.373713 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 364/718] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 30 09:23:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:01.380100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:01.509968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47878cfc-7207-47b0-8445-2b93d73df281 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:01.525079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:01.550411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cb11d-2916-44da-8c07-639d6701c897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:01.551157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cb11d-2916-44da-8c07-639d6701c897 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:01.551438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cb11d-2916-44da-8c07-639d6701c897 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:01.551721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cb11d-2916-44da-8c07-639d6701c897 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:01.552364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cb11d-2916-44da-8c07-639d6701c897 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:01.552737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cb11d-2916-44da-8c07-639d6701c897 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:01.554225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cb11d-2916-44da-8c07-639d6701c897 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:01.560233 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 355/719] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 30 09:23:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:02.906686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85e2f9-ec9d-4457-8525-50244f5fa700 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:02.907226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85e2f9-ec9d-4457-8525-50244f5fa700 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:02.907226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85e2f9-ec9d-4457-8525-50244f5fa700 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:02.909042 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 365/720] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:02.914672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2687-ca30-4e36-a176-ef667d936b4d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:02.915159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2687-ca30-4e36-a176-ef667d936b4d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:02.915235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf2687-ca30-4e36-a176-ef667d936b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:02.938217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-67cf2687-ca30-4e36-a176-ef667d936b4d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:02.939535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-67cf2687-ca30-4e36-a176-ef667d936b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:03.030517 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 356/721] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:23:03.089947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cb72-0e71-4304-a00f-263d92965241 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:03.090274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cb72-0e71-4304-a00f-263d92965241 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:03.090494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cb72-0e71-4304-a00f-263d92965241 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:03.092098 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 366/722] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:03.097475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedb54a-2f05-4ee3-98b1-cca8bb52cf72 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:03.097842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedb54a-2f05-4ee3-98b1-cca8bb52cf72 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:03.098112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedb54a-2f05-4ee3-98b1-cca8bb52cf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:03.126958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-faedb54a-2f05-4ee3-98b1-cca8bb52cf72 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:03.128369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-faedb54a-2f05-4ee3-98b1-cca8bb52cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:03.224635 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 357/723] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:23:03.369017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7cc24-df10-4447-a7b2-cd84caccc7f6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:03.369705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7cc24-df10-4447-a7b2-cd84caccc7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:03.370186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7cc24-df10-4447-a7b2-cd84caccc7f6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:03.372005 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 367/724] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:03.377239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963e110-80cd-4fbe-951f-7dad9cbec7fc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:03.377972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963e110-80cd-4fbe-951f-7dad9cbec7fc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:03.378734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963e110-80cd-4fbe-951f-7dad9cbec7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:03.400613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0963e110-80cd-4fbe-951f-7dad9cbec7fc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:03.402868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0963e110-80cd-4fbe-951f-7dad9cbec7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:03.435232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:03.477149 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 358/725] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:23:03.504035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7cc24-df10-4447-a7b2-cd84caccc7f6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:03.592233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:03.637501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03f3b-f139-47a2-8d5b-6255a6fffaa5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:03.637713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03f3b-f139-47a2-8d5b-6255a6fffaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:03.637967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03f3b-f139-47a2-8d5b-6255a6fffaa5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:03.639524 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 359/726] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:03.640210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe03f3b-f139-47a2-8d5b-6255a6fffaa5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:03.644123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7cc24-df10-4447-a7b2-cd84caccc7f6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:03.681644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b03e2-3633-405b-afe2-c52ea77ea6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:03.682285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b03e2-3633-405b-afe2-c52ea77ea6ae None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:03.682571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b03e2-3633-405b-afe2-c52ea77ea6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:03.682848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b03e2-3633-405b-afe2-c52ea77ea6ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:03.696325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b03e2-3633-405b-afe2-c52ea77ea6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1a6b03e2-3633-405b-afe2-c52ea77ea6ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:03.701263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:03.702217 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:03.703066 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 368/727] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:03.704288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22e721-cab6-4d06-a436-29fd83356395 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:03.704288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22e721-cab6-4d06-a436-29fd83356395 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:03.704516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22e721-cab6-4d06-a436-29fd83356395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:03.723694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cc22e721-cab6-4d06-a436-29fd83356395 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:03.724898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cc22e721-cab6-4d06-a436-29fd83356395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:03.776518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe03f3b-f139-47a2-8d5b-6255a6fffaa5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:03.796584 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 369/728] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:23:03.796768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22e721-cab6-4d06-a436-29fd83356395 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:03.841171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59b3e7-92a3-47dd-8c98-8118afef6639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:03.841921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59b3e7-92a3-47dd-8c98-8118afef6639 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:03.842275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59b3e7-92a3-47dd-8c98-8118afef6639 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:03.842610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59b3e7-92a3-47dd-8c98-8118afef6639 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:03.843334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59b3e7-92a3-47dd-8c98-8118afef6639 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:03.844826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59b3e7-92a3-47dd-8c98-8118afef6639 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:03.864856 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 360/729] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:23:03 2022] POST /identity/v3/services => generated 240 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:03.865823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59b3e7-92a3-47dd-8c98-8118afef6639 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:03.883713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:03.934102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:03.977663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22e721-cab6-4d06-a436-29fd83356395 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.067855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.068818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.069306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.069807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.070665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:04.071080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59b3e7-92a3-47dd-8c98-8118afef6639 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.071375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:04.073207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:04.086418 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 370/730] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:04.092875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.119643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca949b-e025-4b8c-9e28-58dca312d7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.120640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca949b-e025-4b8c-9e28-58dca312d7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.121109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca949b-e025-4b8c-9e28-58dca312d7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.121566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca949b-e025-4b8c-9e28-58dca312d7e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.133786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca949b-e025-4b8c-9e28-58dca312d7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-44ca949b-e025-4b8c-9e28-58dca312d7e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:04.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:04.139885 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:04.140727 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:04.143428 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 361/731] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:04.149492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca949b-e025-4b8c-9e28-58dca312d7e3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.225055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d7310-66ea-432f-b793-753e71e6ad76 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.235760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:04.243106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:04.262336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca949b-e025-4b8c-9e28-58dca312d7e3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.266260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.267150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.267659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.268133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.282737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:04.285613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:04.297419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6b79-1873-47fa-8941-73281bebefa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.298077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6b79-1873-47fa-8941-73281bebefa0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.298357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6b79-1873-47fa-8941-73281bebefa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.298630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6b79-1873-47fa-8941-73281bebefa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:04.305641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:04.306786 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:04.307682 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:04.307682 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:04.307682 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:04.308139 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 371/732] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:04.308926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.313668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be6b79-1873-47fa-8941-73281bebefa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-93be6b79-1873-47fa-8941-73281bebefa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:04.317148 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:04.318155 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:04.319442 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 362/733] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:04.320344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b2b22-d571-4bb7-bca2-1929347c4983 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.320530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b2b22-d571-4bb7-bca2-1929347c4983 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.321677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b2b22-d571-4bb7-bca2-1929347c4983 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.323616 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 363/734] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:04.323755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b2b22-d571-4bb7-bca2-1929347c4983 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.428315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b2ef-6597-4aa3-9701-7b3fd7f2c01b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.432488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b2b22-d571-4bb7-bca2-1929347c4983 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.467163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.467914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.468200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.468473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.469140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:04.469447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:04.469619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1805964-0b65-43a3-8938-eaa13bdc422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.470224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1805964-0b65-43a3-8938-eaa13bdc422c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.470504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1805964-0b65-43a3-8938-eaa13bdc422c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.470777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1805964-0b65-43a3-8938-eaa13bdc422c None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.470891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:04.471453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1805964-0b65-43a3-8938-eaa13bdc422c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:04.471890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1805964-0b65-43a3-8938-eaa13bdc422c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:04.473577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1805964-0b65-43a3-8938-eaa13bdc422c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:04.485757 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 372/735] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:04.485757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.490650 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 364/736] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:04.491760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e20b9-6ff1-4e40-b9db-88f54cfe52df None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.492041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e20b9-6ff1-4e40-b9db-88f54cfe52df None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.492312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e20b9-6ff1-4e40-b9db-88f54cfe52df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.498772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:04.513728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-081e20b9-6ff1-4e40-b9db-88f54cfe52df None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:04.514932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-081e20b9-6ff1-4e40-b9db-88f54cfe52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:04.521238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:04.592834 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 365/737] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:04 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 30 09:23:04.593600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e20b9-6ff1-4e40-b9db-88f54cfe52df None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.609982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0a6f-326c-4a7c-b9f2-ff097b7eb7da None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.647770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.648878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.649367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.649878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.650846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:04.651627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:04.653350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:04.663534 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 373/738] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:04.664681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.731311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e20b9-6ff1-4e40-b9db-88f54cfe52df None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.767718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:04.776397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:04.782735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.783413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.783638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.784030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.790513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a531e0-6e05-49e9-8a04-3b26fbece590 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.796731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:04.798231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:04.816807 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:04.817848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:04.818754 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:04.818754 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:04.818754 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:04.821763 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 366/739] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:04.822796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.839375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.840740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.841540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.842114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.852568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:04.854781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:04.878393 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:04.879791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:23:04.880763 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:04.884604 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 374/740] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:04.885802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:04.950590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c588ae-2938-4838-8771-2e14d23677c3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:04.986273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:04.987184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:04.987482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:04.987750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:04.999040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.000503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.001281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044dbb3-4fe4-450a-a8d6-0188adac2a77 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:05.015293 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:05.016232 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:05.017111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:05.017111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:05.017111 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:05.017111 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 367/741] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:05.017111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.043537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.044138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.044418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.044690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.045326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:05.045646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.047009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.053022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:05.060566 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 375/742] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:05.061777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.067440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:05.126499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4366d-1b9d-43c1-8fcd-7ab1857046f3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.165822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256077a-6161-41a1-9e36-8b15ef3d13de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.166454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256077a-6161-41a1-9e36-8b15ef3d13de None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.166732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256077a-6161-41a1-9e36-8b15ef3d13de None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.167005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256077a-6161-41a1-9e36-8b15ef3d13de None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.167652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256077a-6161-41a1-9e36-8b15ef3d13de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.168938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e04473-cadf-497c-8b8e-9f1896ffb1a0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.169048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256077a-6161-41a1-9e36-8b15ef3d13de None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.185677 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 368/743] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:23:05 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:05.185677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256077a-6161-41a1-9e36-8b15ef3d13de None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.228168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.228964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.229334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.229731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.230565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:05.231022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.233730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.263752 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 376/744] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:05.263752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.322059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:05.331242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256077a-6161-41a1-9e36-8b15ef3d13de None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.342347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:05.371817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.372467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.372751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.373091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.373697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:05.374003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.376244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.394537 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 369/745] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:05.394992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.396212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacbb9c-cde0-4fb3-a1bf-91c5dc469c6e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.439220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.439862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.440916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.441204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.450121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.452286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:05.467928 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:05.468895 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:05.469887 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 377/746] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:05.471306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.516921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d9b70-daf8-4175-b0a9-3c802b8714e4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.561894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7660dd-4e58-4634-a960-15b7e67585ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.562762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7660dd-4e58-4634-a960-15b7e67585ff None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.563040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7660dd-4e58-4634-a960-15b7e67585ff None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.563308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7660dd-4e58-4634-a960-15b7e67585ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.586146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86104605-5bbb-4a98-b044-52b286fa5b31 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.591510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:05.615791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:05.639423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.640833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.641188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.641535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.642962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7660dd-4e58-4634-a960-15b7e67585ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.645064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7660dd-4e58-4634-a960-15b7e67585ff None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.651679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.653240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:05.669661 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:05.670663 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:05.671612 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:05.675894 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 378/747] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:05.677356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.777798 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 370/748] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:23:05.779029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7660dd-4e58-4634-a960-15b7e67585ff None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.802714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a82af4-a4d5-4a33-be88-8dbeb5e91ddd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.842155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.843128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.843603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.844184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.845132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:05.845659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.847448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.847968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:05.873283 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 379/749] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:05.873283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.905079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7660dd-4e58-4634-a960-15b7e67585ff None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:05.922002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:05.949355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:05.950005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:05.950285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:05.950554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:05.951442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:05.951977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:05.953998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:05.977723 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 371/750] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:05.978619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:05.999057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af337ab-caa7-40a2-ab7c-d6cf54ea3d41 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:06.045328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:06.046132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.046528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.046799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.061682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:06.063665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:06.085009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:06.086144 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:06.087379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:06.087379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:06.087379 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:06.087379 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 380/751] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:06.090665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:06.118741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123063f3-4de7-4324-ad56-f11f325d5123 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:06.138327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:06.169367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:06.169946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.170461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.170744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.184768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:06.186786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:06.202939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:06.209358 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:06.210359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:06.211246 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:06.211246 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:06.211246 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:06.211246 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 372/752] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:06.211950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:06.223207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ddf6d-1e08-4745-b355-ceaa4e9a110b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:06.270908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa8f5d-e40b-4d35-a64e-3e5cff8aeed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:06.271517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8f5d-e40b-4d35-a64e-3e5cff8aeed6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.271798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8f5d-e40b-4d35-a64e-3e5cff8aeed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.272067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8f5d-e40b-4d35-a64e-3e5cff8aeed6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.318393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a2aab-c1cc-4339-ab76-1127c4881563 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:06.345159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa8f5d-e40b-4d35-a64e-3e5cff8aeed6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:06.347344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa8f5d-e40b-4d35-a64e-3e5cff8aeed6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:06.355019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:06.355616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.355899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.356173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.357108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:06.357108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:06.358499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:06.375243 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 373/753] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:06.376094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:06.411641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:06.444508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:06.461123 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 381/754] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:23:06.462557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c6a8c-d416-43ad-b1ba-fba53ed1fce3 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.462940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c6a8c-d416-43ad-b1ba-fba53ed1fce3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.463726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c6a8c-d416-43ad-b1ba-fba53ed1fce3 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.465729 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 382/755] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:06.466550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c6a8c-d416-43ad-b1ba-fba53ed1fce3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:06.489559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8128332-3b28-4d21-b0d4-208eb855bbb1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:06.525196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6187d3-689f-4445-97f2-f2bd5e960b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:06.525827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6187d3-689f-4445-97f2-f2bd5e960b2f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.526365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6187d3-689f-4445-97f2-f2bd5e960b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.526634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6187d3-689f-4445-97f2-f2bd5e960b2f None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.527271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6187d3-689f-4445-97f2-f2bd5e960b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:06.527569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6187d3-689f-4445-97f2-f2bd5e960b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:06.528936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6187d3-689f-4445-97f2-f2bd5e960b2f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:06.541133 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 374/756] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:06.542353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60ab0b-5f0a-47e1-8e01-24b7e5a709f4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.542619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60ab0b-5f0a-47e1-8e01-24b7e5a709f4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.542889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60ab0b-5f0a-47e1-8e01-24b7e5a709f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.562348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0b60ab0b-5f0a-47e1-8e01-24b7e5a709f4 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:06.563542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0b60ab0b-5f0a-47e1-8e01-24b7e5a709f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:06.595048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c6a8c-d416-43ad-b1ba-fba53ed1fce3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:06.630553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:06.631199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.631474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.631741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.639277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:06.640736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:06.653437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:06.654483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 383/757] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:06.661378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:06.714270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:06.765658 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 375/758] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:06.765658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60ab0b-5f0a-47e1-8e01-24b7e5a709f4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:06.778757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57893fc-e531-47ea-a70a-b2a05a36107a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:06.785335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:06.817211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:06.818302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.818687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.819064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.831904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:06.833785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:06.850488 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:06.851483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:06.852383 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:06.852383 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:06.852383 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:06.852383 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 384/759] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:06.856771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:06.880564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60ab0b-5f0a-47e1-8e01-24b7e5a709f4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:06.930407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:06.931615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:06.932167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:06.933213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:06.943082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:06.945310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:06.960931 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:06.962465 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:06.963544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:06.963544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:06.963544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:06.963544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:06.963544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:06.963544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:06.963544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:06.963544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:06.966130 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 376/760] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:06.967115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:06.977883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:06.981699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446b300-da36-4edb-9a8d-6e3d71d08f7a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.030944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.032105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.032542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.033372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.034882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:07.035361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.037486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.053244 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 385/761] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:07.054814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.062995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:07.089261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd1336-eb7f-431b-a83a-cbacee1e9e81 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.124733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.125687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.126117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.126408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.127042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:07.127347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.129267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.140871 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 377/762] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:07.141499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.169740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe0713-3333-4f6e-b8db-527ad8580a78 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.206432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.207185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.207656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.207990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.208683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:07.209037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.210493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.228311 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 386/763] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:07.228311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.247411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851221f0-2c2f-4fb8-8354-3750ad406f5e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.250646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:07.285905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.286509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.286786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.287073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.287784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=60761ed7c27f4018a6b4791b3b060bb0) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:07.288085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.289524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.292377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:07.295640 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 378/764] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:07 2022] GET /identity/v3/role_assignments?user.id=60761ed7c27f4018a6b4791b3b060bb0&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:07.296525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.372003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05ac68-bc28-4387-96e7-bfe1933eb61d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.426287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fe7c0-9ccf-44a7-98dc-b0eda94e4915 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.438831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.440618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.441099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.441539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] PATH_INFO: `/v3/services/network` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.442676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.445188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:07.460884 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:07.462528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:07.464538 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 387/765] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:07 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:07.466402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.471883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d94c3-a8a5-4338-b714-b69726c4db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.472529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d94c3-a8a5-4338-b714-b69726c4db5d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.472809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d94c3-a8a5-4338-b714-b69726c4db5d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.473085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d94c3-a8a5-4338-b714-b69726c4db5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.480641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d94c3-a8a5-4338-b714-b69726c4db5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.482072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d94c3-a8a5-4338-b714-b69726c4db5d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0a4d94c3-a8a5-4338-b714-b69726c4db5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:07.495066 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:07.495966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:07.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 379/766] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:07.498084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6657-990b-4d70-b1e8-3e226cff26a5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.498084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6657-990b-4d70-b1e8-3e226cff26a5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.498084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e6657-990b-4d70-b1e8-3e226cff26a5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.501881 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 380/767] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:07.501881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e6657-990b-4d70-b1e8-3e226cff26a5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.561154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:07.567798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:07.581525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af43864-4c72-4f0a-9054-7b7014f721bc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.615793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e6657-990b-4d70-b1e8-3e226cff26a5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.618694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f412cf-660d-4c57-8d77-e089473beb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.619355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f412cf-660d-4c57-8d77-e089473beb17 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.619634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f412cf-660d-4c57-8d77-e089473beb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.619906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f412cf-660d-4c57-8d77-e089473beb17 None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/aa055ee122ce4e92b4da7f8222521a9e/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.621865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f412cf-660d-4c57-8d77-e089473beb17 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=aa055ee122ce4e92b4da7f8222521a9e, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.623292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f412cf-660d-4c57-8d77-e089473beb17 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.637885 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 388/768] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:23:07 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/aa055ee122ce4e92b4da7f8222521a9e/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:23:07.638723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f412cf-660d-4c57-8d77-e089473beb17 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.654988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b222-9f54-423e-b286-ccbcd58bb970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.656020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b222-9f54-423e-b286-ccbcd58bb970 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.656522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b222-9f54-423e-b286-ccbcd58bb970 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.657011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b222-9f54-423e-b286-ccbcd58bb970 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.657941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b222-9f54-423e-b286-ccbcd58bb970 None admin] RBAC: Adding query filter params (name=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:07.658352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b222-9f54-423e-b286-ccbcd58bb970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.660173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b222-9f54-423e-b286-ccbcd58bb970 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.669670 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 381/769] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:07.669670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afbf37-5227-40e5-8ec4-272e2c12ee93 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.669670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afbf37-5227-40e5-8ec4-272e2c12ee93 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.669670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afbf37-5227-40e5-8ec4-272e2c12ee93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.689372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-06afbf37-5227-40e5-8ec4-272e2c12ee93 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:07.690852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-06afbf37-5227-40e5-8ec4-272e2c12ee93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:07.786463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f412cf-660d-4c57-8d77-e089473beb17 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.798396 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 382/770] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:07.800098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afbf37-5227-40e5-8ec4-272e2c12ee93 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.826745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:07.827315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:07.827599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:07.827872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:07.828634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:07.828942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:07.830869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:07.844309 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 389/771] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:07.854733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:07.859606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:07.913673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:07.964276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afbf37-5227-40e5-8ec4-272e2c12ee93 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:07.978609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f7097-76e1-4105-9209-17c6e73f3aaf None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:08.005048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:08.005881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:08.006357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:08.006728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:08.007458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] RBAC: Adding query filter params (type=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:08.007876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:08.009541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:08.015403 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 383/772] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:08.017215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:08.018617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:08.018617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:08.018795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:08.019588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=29179e1dc1b045a08cbd7feadc2ececd) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:08.019902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:08.020979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:08.021367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:08.028950 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 390/773] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:07 2022] GET /identity/v3/role_assignments?user.id=29179e1dc1b045a08cbd7feadc2ececd&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:08.133086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:08.134703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deb957-60bd-4f23-8b82-257f9a50e60b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:08.155133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:08.185810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:08.186447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:08.186722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:08.187408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:08.188033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] RBAC: Adding query filter params (interface=public, service_id=5a556c7941fa4fd19841b5528ed803ce, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:08.188336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:08.189771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:08.211450 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 384/774] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 30 09:23:08 2022] GET /identity/v3/endpoints?service_id=5a556c7941fa4fd19841b5528ed803ce&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:08.217310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:08.283021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:08.335733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb98e73-e11a-4fcb-8a93-5b1a1319b2dd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:08.351359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3d900-cd3f-4b3a-8953-becd68a74557 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:08.370706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e188c6-312d-4c2e-97a8-d51834da40c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:08.371311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e188c6-312d-4c2e-97a8-d51834da40c5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:08.371588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e188c6-312d-4c2e-97a8-d51834da40c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:08.371856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e188c6-312d-4c2e-97a8-d51834da40c5 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:08.372457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e188c6-312d-4c2e-97a8-d51834da40c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:08.373853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e188c6-312d-4c2e-97a8-d51834da40c5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:08.379703 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 385/775] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:23:08 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:08.412116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:08.413118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:08.413585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:08.414088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:08.414888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:08.416464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:08.427240 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:08.428490 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:08.429531 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:08.431997 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 391/776] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 30 09:23:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:08.438098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e188c6-312d-4c2e-97a8-d51834da40c5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:08.502707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:08.582741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e188c6-312d-4c2e-97a8-d51834da40c5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:08.634699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:08.635314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:08.635600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:08.635876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:08.636511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:08.636816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:08.638233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:08.646989 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 386/777] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 30 09:23:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:08.652923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:08.798923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69499f-8564-449e-8648-fae711966fa4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:08.833876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:08.843959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:08.844625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:08.844908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:08.845179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:08.845847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:08.846672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:08.848184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:08.853613 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 392/778] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 30 09:23:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:08.859315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:09.014901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8b303-bd30-4b84-ac95-12a7001906aa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:09.031137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:09.064905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f11c12-6eb9-4fed-aa4f-b7cc15f61bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:09.064905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11c12-6eb9-4fed-aa4f-b7cc15f61bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:09.064905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11c12-6eb9-4fed-aa4f-b7cc15f61bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:09.064905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f11c12-6eb9-4fed-aa4f-b7cc15f61bc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:09.065793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f11c12-6eb9-4fed-aa4f-b7cc15f61bc3 None admin] RBAC: Adding query filter params (interface=public, service_id=4e544d36b578407cb237b36685d8bd77, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:09.066115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f11c12-6eb9-4fed-aa4f-b7cc15f61bc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:09.067508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f11c12-6eb9-4fed-aa4f-b7cc15f61bc3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:09.077030 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 387/779] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 30 09:23:08 2022] GET /identity/v3/endpoints?service_id=4e544d36b578407cb237b36685d8bd77&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:09.082968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:09.225300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa49d4a-4444-435e-9400-5161e485aa8a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:09.273210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922079e-ed21-48d6-8d0f-966577bcbf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:09.273858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922079e-ed21-48d6-8d0f-966577bcbf31 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:09.274140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922079e-ed21-48d6-8d0f-966577bcbf31 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:09.274413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922079e-ed21-48d6-8d0f-966577bcbf31 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:09.275086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922079e-ed21-48d6-8d0f-966577bcbf31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:09.276768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922079e-ed21-48d6-8d0f-966577bcbf31 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:09.282603 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 393/780] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:23:09 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:09.741854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d1ff8-3a71-48ea-b76f-b1f4cbdc9bb5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:09.742206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d1ff8-3a71-48ea-b76f-b1f4cbdc9bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:09.742327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d1ff8-3a71-48ea-b76f-b1f4cbdc9bb5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:09.743962 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 388/781] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:09.748898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ca24b-e86c-44c9-8787-a2cd95141340 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:09.749195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ca24b-e86c-44c9-8787-a2cd95141340 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:09.749471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ca24b-e86c-44c9-8787-a2cd95141340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:09.772979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-851ca24b-e86c-44c9-8787-a2cd95141340 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:09.774263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-851ca24b-e86c-44c9-8787-a2cd95141340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:09.853767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:09.867593 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 394/782] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:10.282508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d1ff8-3a71-48ea-b76f-b1f4cbdc9bb5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:10.340555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89531f1-7c76-4707-a32d-46123685daa1 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:10.341524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89531f1-7c76-4707-a32d-46123685daa1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:10.342042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89531f1-7c76-4707-a32d-46123685daa1 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:10.343853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 395/783] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:10.350132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46d6a-2857-4f26-baf1-52638d53a33e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:10.350681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46d6a-2857-4f26-baf1-52638d53a33e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:10.351188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46d6a-2857-4f26-baf1-52638d53a33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:10.379390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6bc46d6a-2857-4f26-baf1-52638d53a33e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:10.384659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6bc46d6a-2857-4f26-baf1-52638d53a33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:10.460420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d1ff8-3a71-48ea-b76f-b1f4cbdc9bb5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:10.531261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23322-c808-4536-8462-5854d18f8d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:10.531915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23322-c808-4536-8462-5854d18f8d98 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:10.532195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23322-c808-4536-8462-5854d18f8d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:10.532467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23322-c808-4536-8462-5854d18f8d98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:10.539324 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 396/784] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:10.547797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb23322-c808-4536-8462-5854d18f8d98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4cb23322-c808-4536-8462-5854d18f8d98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:10.553480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:10.554409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:10.555252 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 389/785] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:10.567260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46d6a-2857-4f26-baf1-52638d53a33e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:10.691751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46d6a-2857-4f26-baf1-52638d53a33e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:10.721388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:10.740612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a9f03-5647-4b24-87fc-8e5d38cbf395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:10.741245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9f03-5647-4b24-87fc-8e5d38cbf395 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:10.741525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9f03-5647-4b24-87fc-8e5d38cbf395 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:10.741835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a9f03-5647-4b24-87fc-8e5d38cbf395 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:10.742524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a9f03-5647-4b24-87fc-8e5d38cbf395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:10.742829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a9f03-5647-4b24-87fc-8e5d38cbf395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:10.744211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a9f03-5647-4b24-87fc-8e5d38cbf395 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:10.757697 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 397/786] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:10.763233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23322-c808-4536-8462-5854d18f8d98 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:10.813507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:10.885993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc41da1-7571-41fa-bb33-bc7e3c21db12 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:10.886310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc41da1-7571-41fa-bb33-bc7e3c21db12 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:10.886546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc41da1-7571-41fa-bb33-bc7e3c21db12 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:10.892400 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 398/787] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:10.893960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705f3aa-2897-43dd-a827-142aceb813ae None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:10.894247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705f3aa-2897-43dd-a827-142aceb813ae None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:10.894525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705f3aa-2897-43dd-a827-142aceb813ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:10.906898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23322-c808-4536-8462-5854d18f8d98 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:10.934694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b705f3aa-2897-43dd-a827-142aceb813ae None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:10.935902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b705f3aa-2897-43dd-a827-142aceb813ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:10.945803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:10.946705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:10.947184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:10.947656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:10.959399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:10.961205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:10.976525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:10.977537 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:10.978471 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:10.978471 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:10.978471 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:10.980564 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 390/788] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:10.985809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.051083 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 399/789] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:11.052433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705f3aa-2897-43dd-a827-142aceb813ae None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.137988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:11.161774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4e7c9-3984-4df2-b285-84d25343df7f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:11.210283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:11.210878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.211156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.212199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.212888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:11.213776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:11.215382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:11.218556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705f3aa-2897-43dd-a827-142aceb813ae None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:11.240738 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 391/790] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:11.241362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:11.248537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.279981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10cc3b-e5b3-44e3-b074-5a74a1f675f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:11.280619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10cc3b-e5b3-44e3-b074-5a74a1f675f6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.280896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10cc3b-e5b3-44e3-b074-5a74a1f675f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.281170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10cc3b-e5b3-44e3-b074-5a74a1f675f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.306915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10cc3b-e5b3-44e3-b074-5a74a1f675f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2f10cc3b-e5b3-44e3-b074-5a74a1f675f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:11.320126 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:11.321146 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:11.322027 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 400/791] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:11.335861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10cc3b-e5b3-44e3-b074-5a74a1f675f6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.396154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ff7bb-4bb1-4e9a-a93b-ad9d3bd8f32c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:11.433401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:11.433936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.434211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.434477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.442517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:11.444038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:11.456226 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:11.457218 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:23:11.458180 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:11.461852 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 392/792] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:11.462698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.473102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:11.509039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10cc3b-e5b3-44e3-b074-5a74a1f675f6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:11.546224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:11.546837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.547119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.547395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.548062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:11.548531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:11.550024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:11.558381 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 401/793] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:11.559014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.577051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9eb00e-90fb-4ac1-8963-3924c2dc2821 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:11.595396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:11.611880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93902593-ce6f-440f-9ee1-5b15fd65dabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:11.612506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93902593-ce6f-440f-9ee1-5b15fd65dabe None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.612791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93902593-ce6f-440f-9ee1-5b15fd65dabe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.613063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93902593-ce6f-440f-9ee1-5b15fd65dabe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.623379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93902593-ce6f-440f-9ee1-5b15fd65dabe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-93902593-ce6f-440f-9ee1-5b15fd65dabe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:11.629331 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:11.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:11.631198 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 393/794] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:11.632305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abded2-0b60-4776-b88a-7a940f421028 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.632405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abded2-0b60-4776-b88a-7a940f421028 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.632604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abded2-0b60-4776-b88a-7a940f421028 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.635453 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 394/795] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:11.635453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abded2-0b60-4776-b88a-7a940f421028 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.671580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac113-155a-4ec1-a230-6422f740b1dd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:11.705457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:11.706088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.706371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.706875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.707362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:11.707674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:11.709153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:11.710651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:11.733043 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 402/796] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:11.733043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.754847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abded2-0b60-4776-b88a-7a940f421028 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:11.794161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33c927-5bcb-47a9-857a-ebef68dba496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:11.795020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33c927-5bcb-47a9-857a-ebef68dba496 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.795418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33c927-5bcb-47a9-857a-ebef68dba496 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.795692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33c927-5bcb-47a9-857a-ebef68dba496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.807426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33c927-5bcb-47a9-857a-ebef68dba496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:11.808930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33c927-5bcb-47a9-857a-ebef68dba496 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1f33c927-5bcb-47a9-857a-ebef68dba496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:11.826922 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:11.827970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:11.828855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:11.828855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:11.828855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:11.828855 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 395/797] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:11.829554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e3be-32f5-4390-8108-055fdee99349 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.829554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e3be-32f5-4390-8108-055fdee99349 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.829848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41e3be-32f5-4390-8108-055fdee99349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.833802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:11.844404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f5407-4b58-4d59-907b-0d167f70818e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:11.849502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fe41e3be-32f5-4390-8108-055fdee99349 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:11.850732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fe41e3be-32f5-4390-8108-055fdee99349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:11.881967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:11.882726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:11.883312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:11.883588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:11.884620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:11.884925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:11.886322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:11.894682 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 403/798] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:11.895482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.927367 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 396/799] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:11.928435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41e3be-32f5-4390-8108-055fdee99349 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:11.979663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.038448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41e3be-32f5-4390-8108-055fdee99349 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.076355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.076980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.077261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.077534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.078222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:12.078529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.079927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.098022 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 397/800] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:12.098022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.125794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53087dcc-973e-4eb1-af8e-e14ddb222e15 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.161094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95007d-1b73-4ad7-b77e-d2793b78802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.161754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95007d-1b73-4ad7-b77e-d2793b78802d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.162033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95007d-1b73-4ad7-b77e-d2793b78802d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.162320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95007d-1b73-4ad7-b77e-d2793b78802d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.173658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95007d-1b73-4ad7-b77e-d2793b78802d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.176071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95007d-1b73-4ad7-b77e-d2793b78802d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-db95007d-1b73-4ad7-b77e-d2793b78802d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:12.191103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:12.192116 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:12.193540 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:12.193540 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:12.193540 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:12.193540 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 404/801] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:12.193823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0738-dd1e-44b8-b0b7-dcd0f775f3e4 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.193891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0738-dd1e-44b8-b0b7-dcd0f775f3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.194153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd0738-dd1e-44b8-b0b7-dcd0f775f3e4 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.195513 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 405/802] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:12.200042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0738-dd1e-44b8-b0b7-dcd0f775f3e4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.208841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c649c86-a101-4eba-b961-f23ac0fe7460 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.209887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.218263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.243961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.244581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.244860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.245130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.255771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.257224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:12.272182 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:12.273295 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:12.274228 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:12.274228 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:12.274228 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:12.274228 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 398/803] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:12.274462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.308612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd0738-dd1e-44b8-b0b7-dcd0f775f3e4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.341839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8903e78-b7ed-48ee-98ab-1055f5c6c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.342468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903e78-b7ed-48ee-98ab-1055f5c6c95e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.342746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903e78-b7ed-48ee-98ab-1055f5c6c95e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.343018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8903e78-b7ed-48ee-98ab-1055f5c6c95e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.350424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8903e78-b7ed-48ee-98ab-1055f5c6c95e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.351852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8903e78-b7ed-48ee-98ab-1055f5c6c95e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d8903e78-b7ed-48ee-98ab-1055f5c6c95e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:12.363548 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:12.364532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:12.365504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:12.369261 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 406/804] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:12.370641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b788-f6ea-4c50-812c-5bda3d52f3d9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.370917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b788-f6ea-4c50-812c-5bda3d52f3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.371181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b788-f6ea-4c50-812c-5bda3d52f3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.374933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dda3f8-6ba8-4140-9478-7a9585e2505d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.388652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fb39b788-f6ea-4c50-812c-5bda3d52f3d9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:12.389877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fb39b788-f6ea-4c50-812c-5bda3d52f3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:12.412895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.413506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.413828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.414110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.414751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:12.415058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.416446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.428637 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 399/805] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:12.429651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.450414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.452516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.461773 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 407/806] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:12.462424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39b788-f6ea-4c50-812c-5bda3d52f3d9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.538662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16695c5-a67d-4457-b8b0-8e3e85f202f6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.572075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39b788-f6ea-4c50-812c-5bda3d52f3d9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.573916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.574891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.574891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.575083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.575658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.577030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:12.586820 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:12.587855 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:12.588685 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 400/807] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:12.590843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.620678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.621775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.622072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.622343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.622987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:12.623282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.624637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.636212 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 408/808] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:12.637060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.687746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.697544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.707466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501245ba-f27f-4ca2-8d35-916fb49635c1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.744941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.745550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.745852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.746123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.746847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:12.747142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.748568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.750106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ed2a4-3910-4adf-b287-789b2bb3882a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.766410 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 401/809] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:12.767219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.788173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.788795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.789069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.789342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.795984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.797399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:12.808867 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:12.809957 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 409/810] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:12.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.864463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cadaf-cc91-45b2-83cf-af8be116e1bd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.897399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.898028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.898307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.898579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.899198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:12.899504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.901128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.905781 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 402/811] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:12.906430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:12.914497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.917388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439fd88-29aa-43cb-9ab6-6f28d0f7d5fa None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:12.933453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:12.952262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:12.952890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:12.953167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:12.953435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:12.961306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:12.962710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:12.973601 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:12.974751 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 410/812] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:12.975779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.011918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531324c2-d8f1-4f43-8b4c-3d57a426a2b2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.046773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.047396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.047678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.047948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.048524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.050566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:13.059223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:13.060181 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:13.061035 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 403/813] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:13.062225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.075181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e50001-5c80-4fd5-a8dc-3a01a3707385 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.107875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524d1ea-a0e0-4960-82ed-13356aeef9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.108473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524d1ea-a0e0-4960-82ed-13356aeef9db None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.108751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524d1ea-a0e0-4960-82ed-13356aeef9db None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.109027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524d1ea-a0e0-4960-82ed-13356aeef9db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.128960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:13.154508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:13.161024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35466e-19d5-4079-8a73-fe7537e72bc9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.177022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524d1ea-a0e0-4960-82ed-13356aeef9db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.178506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524d1ea-a0e0-4960-82ed-13356aeef9db None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.195485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.196085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.196362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.196633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.197321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:13.197622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.199070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.210734 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 404/814] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:13.211593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.276191 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 411/815] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:23:13.277041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524d1ea-a0e0-4960-82ed-13356aeef9db None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.311428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ffa4e-07f4-4a45-98e7-0441fc3e43da None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.343110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:13.349383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.350015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.350295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.350571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.350686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:13.351192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:13.351531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.353039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.357753 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 405/816] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:13.358518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.378910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524d1ea-a0e0-4960-82ed-13356aeef9db None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.418289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.418897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.419176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.419448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.420172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:13.420475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.421946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.438967 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 412/817] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 30 09:23:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:13.440413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.463834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855870b-e942-4356-8d64-1c94924f0ea7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.498764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.499422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.499971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.499971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.500603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:13.500906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.502397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.506783 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 406/818] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 30 09:23:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:13.507567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.536474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983819e-3139-4ccf-8dc3-2261b5ae4eac None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.567111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:13.570462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e879ad-bbc8-45a9-a1e9-d30ff59e4c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.571121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e879ad-bbc8-45a9-a1e9-d30ff59e4c4a None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.571399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e879ad-bbc8-45a9-a1e9-d30ff59e4c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.571671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e879ad-bbc8-45a9-a1e9-d30ff59e4c4a None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/60761ed7c27f4018a6b4791b3b060bb0/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.573565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e879ad-bbc8-45a9-a1e9-d30ff59e4c4a None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=60761ed7c27f4018a6b4791b3b060bb0, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.575026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e879ad-bbc8-45a9-a1e9-d30ff59e4c4a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.577206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:13.588277 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 413/819] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:23:13 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/60761ed7c27f4018a6b4791b3b060bb0/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:23:13.589166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e879ad-bbc8-45a9-a1e9-d30ff59e4c4a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.616813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497c55e-6fbb-44fd-9460-948baba06764 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.662320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.663317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.663754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.664223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.680581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.682388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:13.702484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:13.703541 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:13.704407 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:13.704407 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:13.704407 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:13.704407 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 407/820] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:13.704407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.719614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e879ad-bbc8-45a9-a1e9-d30ff59e4c4a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.764183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7fd00-463f-4917-a747-e6aef37364cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.765255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7fd00-463f-4917-a747-e6aef37364cc None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.765255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7fd00-463f-4917-a747-e6aef37364cc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.765421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7fd00-463f-4917-a747-e6aef37364cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.766497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7fd00-463f-4917-a747-e6aef37364cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.768897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7fd00-463f-4917-a747-e6aef37364cc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.792983 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 414/821] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:23:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:13.794126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7fd00-463f-4917-a747-e6aef37364cc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.823268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1eb676-e4b5-4f42-b49d-ff3738ba3688 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.839617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:13.863071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:13.884035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.884692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.885058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.885336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.901981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:13.904208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:13.928056 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:13.930635 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:13.933062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:13.933062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:13.933062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:13.933062 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 408/822] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:13.933062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:13.947916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7fd00-463f-4917-a747-e6aef37364cc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:13.994084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:13.995038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:13.996379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:13.997262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:13.998133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:13.998558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.001206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.010238 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 415/823] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 30 09:23:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:14.010238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:14.082750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402466-55a8-4d0c-8ebd-0d6cc24bee8f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.120780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.121589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.121975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.122320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.123037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:14.123414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.124964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.136619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59142d91-60a6-4028-a1bf-abfb9e494f83 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.143933 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 409/824] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:14.143933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:14.157786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:14.161958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:14.181836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.182446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.182728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.183004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.183647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:14.184376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.186136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.204914 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 416/825] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:14.204914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:14.256213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6384e5-48c3-450b-b35d-130157ae55da None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.293338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4cb2-bcc1-4abe-a9fd-6c177b31a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.293987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4cb2-bcc1-4abe-a9fd-6c177b31a18d None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.294281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4cb2-bcc1-4abe-a9fd-6c177b31a18d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.294539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4cb2-bcc1-4abe-a9fd-6c177b31a18d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.295210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e4cb2-bcc1-4abe-a9fd-6c177b31a18d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.296589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e4cb2-bcc1-4abe-a9fd-6c177b31a18d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.321113 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 410/826] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:23:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:14.321440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d10de-a709-433c-b310-73ae6a5cc576 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.321974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4cb2-bcc1-4abe-a9fd-6c177b31a18d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:14.375123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.375762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.376041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.376342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.384909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.386566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:14.403969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:14.405124 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:14.406215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:14.406215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:14.406215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:14.406215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:14.406215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:14.406215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:14.406215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:14.406215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:14.410307 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 417/827] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:14.411625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:14.451203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:14.460067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:14.486612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4cb2-bcc1-4abe-a9fd-6c177b31a18d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.536868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.537759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.538200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.538614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.547241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.548912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:14.563552 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:14.564708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:14.566263 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 411/828] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:14.568927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598473b2-d344-440c-9e1d-970c051fe0af None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.570544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:14.613210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.613885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.614131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.614401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.615090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:14.615392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.616820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.633745 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 418/829] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:14.638556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:14.691985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd6a77-5db0-4981-9280-40b9c880effd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.733470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.734353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.734776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.735220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.736055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:14.736633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.738936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.740866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:14.759757 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 412/830] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:14.761059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:14.764428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:14.776727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f666e5-a047-43eb-965b-0b61e2e1bac3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.813798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a93068-7508-4d9f-8263-cbadeae048d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.814328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a93068-7508-4d9f-8263-cbadeae048d5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.814637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a93068-7508-4d9f-8263-cbadeae048d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.815139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a93068-7508-4d9f-8263-cbadeae048d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.816078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a93068-7508-4d9f-8263-cbadeae048d5 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=aa055ee122ce4e92b4da7f8222521a9e) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:14.816401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a93068-7508-4d9f-8263-cbadeae048d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.817870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a93068-7508-4d9f-8263-cbadeae048d5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.829790 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 419/831] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:14 2022] GET /identity/v3/role_assignments?user.id=aa055ee122ce4e92b4da7f8222521a9e&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:14.890312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda6aac-7388-4247-a5a7-298b23ca8cb1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:14.930573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484cf8e-7873-4d4b-b18e-828cf458ddbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:14.931501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484cf8e-7873-4d4b-b18e-828cf458ddbe None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:14.931959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484cf8e-7873-4d4b-b18e-828cf458ddbe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:14.932378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484cf8e-7873-4d4b-b18e-828cf458ddbe None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/29179e1dc1b045a08cbd7feadc2ececd/roles/170f0bd5d6914cb8ad93877cd5cd07bb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:14.951159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484cf8e-7873-4d4b-b18e-828cf458ddbe None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=29179e1dc1b045a08cbd7feadc2ececd, role_id=170f0bd5d6914cb8ad93877cd5cd07bb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:14.952883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484cf8e-7873-4d4b-b18e-828cf458ddbe None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:14.969652 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 413/832] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:23:14 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/29179e1dc1b045a08cbd7feadc2ececd/roles/170f0bd5d6914cb8ad93877cd5cd07bb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:23:15.834188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf7dc1-a17e-4878-8a49-e8f4f26e3bb0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:15.834671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf7dc1-a17e-4878-8a49-e8f4f26e3bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:15.834671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf7dc1-a17e-4878-8a49-e8f4f26e3bb0 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:15.836123 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 420/833] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:15.841481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dc1aa-1b7b-4a0d-84ea-52d337f70077 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:15.842306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dc1aa-1b7b-4a0d-84ea-52d337f70077 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:15.842776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dc1aa-1b7b-4a0d-84ea-52d337f70077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:15.861059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e02dc1aa-1b7b-4a0d-84ea-52d337f70077 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:15.863336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e02dc1aa-1b7b-4a0d-84ea-52d337f70077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:15.908383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:15.936620 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 414/834] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:16.213593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf7dc1-a17e-4878-8a49-e8f4f26e3bb0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:16.323157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf7dc1-a17e-4878-8a49-e8f4f26e3bb0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:16.358571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97324e-f49d-47d7-beb2-ba9c8f538532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:16.359200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97324e-f49d-47d7-beb2-ba9c8f538532 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:16.359477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97324e-f49d-47d7-beb2-ba9c8f538532 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:16.359746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97324e-f49d-47d7-beb2-ba9c8f538532 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:16.370515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97324e-f49d-47d7-beb2-ba9c8f538532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ed97324e-f49d-47d7-beb2-ba9c8f538532 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:16.375274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:16.376190 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:16.377007 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 421/835] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:16.382969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dc1aa-1b7b-4a0d-84ea-52d337f70077 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:16.483766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dc1aa-1b7b-4a0d-84ea-52d337f70077 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:16.523373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef0ace-9f27-48e8-8d94-976ff12b5d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:16.523995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0ace-9f27-48e8-8d94-976ff12b5d47 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:16.524272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0ace-9f27-48e8-8d94-976ff12b5d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:16.524542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0ace-9f27-48e8-8d94-976ff12b5d47 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:16.525217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef0ace-9f27-48e8-8d94-976ff12b5d47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:16.525516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef0ace-9f27-48e8-8d94-976ff12b5d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:16.527195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef0ace-9f27-48e8-8d94-976ff12b5d47 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:16.537348 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 415/836] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:16.543032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97324e-f49d-47d7-beb2-ba9c8f538532 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:16.560082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:16.622206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff21985-d2d9-46fc-b575-339e132fdbcf None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:16.622836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff21985-d2d9-46fc-b575-339e132fdbcf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:16.623329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff21985-d2d9-46fc-b575-339e132fdbcf None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:16.625109 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 416/837] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:16.630389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce601-d754-4645-a468-c5847b57f54a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:16.630907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce601-d754-4645-a468-c5847b57f54a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:16.631377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce601-d754-4645-a468-c5847b57f54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:16.650077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0f7ce601-d754-4645-a468-c5847b57f54a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:16.651847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0f7ce601-d754-4645-a468-c5847b57f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:16.652947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97324e-f49d-47d7-beb2-ba9c8f538532 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:16.682443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:16.696361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:16.697003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:16.697277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:16.697543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:16.711434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:16.713272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:16.729311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:16.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:16.731179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:16.731179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:16.731179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:16.731344 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 417/838] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:16.738232 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 422/839] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:16.741114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ce601-d754-4645-a468-c5847b57f54a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:16.844318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ce601-d754-4645-a468-c5847b57f54a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:16.880482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:16.881099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:16.881382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:16.881668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:16.882323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:16.882629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:16.884126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:16.896777 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 418/840] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:16.902538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:16.972871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:17.011940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc48b4-53cc-4044-9562-324bfb8ecfb5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:17.016489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.053272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:17.069720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:17.070361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.070636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.070902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.082021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:17.083509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:17.100881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:17.102305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:17.103391 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 423/841] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:17.108176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.144047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144a795-5905-41db-b3ea-3bb85dd6717a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:17.179678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786f567-f002-4510-98d9-4b77d35b16b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:17.180305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f567-f002-4510-98d9-4b77d35b16b8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.180581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f567-f002-4510-98d9-4b77d35b16b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.180851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786f567-f002-4510-98d9-4b77d35b16b8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.181423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786f567-f002-4510-98d9-4b77d35b16b8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:17.182822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786f567-f002-4510-98d9-4b77d35b16b8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9786f567-f002-4510-98d9-4b77d35b16b8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:17.192259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:17.193372 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:17.194805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:17.198403 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 419/842] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:17.199089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e640e1-7e5b-45d5-a4d2-1a28541e2ce0 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.199370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e640e1-7e5b-45d5-a4d2-1a28541e2ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.199639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e640e1-7e5b-45d5-a4d2-1a28541e2ce0 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.201673 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 420/843] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:17.201673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e640e1-7e5b-45d5-a4d2-1a28541e2ce0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab3303-858d-4a84-9bb9-c3d041ec19a1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:17.251542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:17.261114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c84c01-d7ec-4a64-8db9-68593d6e814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:17.261750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84c01-d7ec-4a64-8db9-68593d6e814a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.262069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84c01-d7ec-4a64-8db9-68593d6e814a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.262289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c84c01-d7ec-4a64-8db9-68593d6e814a None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.263003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c84c01-d7ec-4a64-8db9-68593d6e814a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:17.263302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c84c01-d7ec-4a64-8db9-68593d6e814a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:17.264711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c84c01-d7ec-4a64-8db9-68593d6e814a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:17.281393 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 424/844] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:17.282593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca17c7-da43-492c-9e38-43744848d911 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.282874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca17c7-da43-492c-9e38-43744848d911 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.283139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca17c7-da43-492c-9e38-43744848d911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.300922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f6ca17c7-da43-492c-9e38-43744848d911 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:17.301840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:17.302122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f6ca17c7-da43-492c-9e38-43744848d911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:17.316813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e640e1-7e5b-45d5-a4d2-1a28541e2ce0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:17.352437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfdaa4-b147-4dec-b359-664f22167831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:17.353040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdaa4-b147-4dec-b359-664f22167831 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.353317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdaa4-b147-4dec-b359-664f22167831 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.353587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfdaa4-b147-4dec-b359-664f22167831 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.354241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfdaa4-b147-4dec-b359-664f22167831 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:17.354538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfdaa4-b147-4dec-b359-664f22167831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:17.355906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfdaa4-b147-4dec-b359-664f22167831 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:17.360370 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 421/845] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:17.362903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6d65-8ac6-4150-a0b2-5083c8ef3b3c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.363188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6d65-8ac6-4150-a0b2-5083c8ef3b3c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.363464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6d65-8ac6-4150-a0b2-5083c8ef3b3c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.364707 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 422/846] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:17.365671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b6d65-8ac6-4150-a0b2-5083c8ef3b3c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.372448 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 425/847] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:17.373154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca17c7-da43-492c-9e38-43744848d911 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.482442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b6d65-8ac6-4150-a0b2-5083c8ef3b3c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:17.484698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca17c7-da43-492c-9e38-43744848d911 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:17.500285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:17.520476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acb8d0-159d-4ece-9cd8-41331cb83b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:17.521010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acb8d0-159d-4ece-9cd8-41331cb83b01 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.521322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acb8d0-159d-4ece-9cd8-41331cb83b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.521557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acb8d0-159d-4ece-9cd8-41331cb83b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.523670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d93d1-cb3a-453a-8a0e-1d22b97fa4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:17.524275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d93d1-cb3a-453a-8a0e-1d22b97fa4cd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.524553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d93d1-cb3a-453a-8a0e-1d22b97fa4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.525523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d93d1-cb3a-453a-8a0e-1d22b97fa4cd None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.525523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d93d1-cb3a-453a-8a0e-1d22b97fa4cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:17.526040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d93d1-cb3a-453a-8a0e-1d22b97fa4cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:17.527355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d93d1-cb3a-453a-8a0e-1d22b97fa4cd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:17.532332 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 426/848] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:17.539962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ece23f-faf2-4870-bece-2dbb1a7ac5b9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.540894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ece23f-faf2-4870-bece-2dbb1a7ac5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.541395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ece23f-faf2-4870-bece-2dbb1a7ac5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.551268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:17.569896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-05ece23f-faf2-4870-bece-2dbb1a7ac5b9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:17.571678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-05ece23f-faf2-4870-bece-2dbb1a7ac5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:17.605149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acb8d0-159d-4ece-9cd8-41331cb83b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:17.606705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acb8d0-159d-4ece-9cd8-41331cb83b01 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:17.664423 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 427/849] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:17.665867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ece23f-faf2-4870-bece-2dbb1a7ac5b9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.735440 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 423/850] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:23:17.738635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acb8d0-159d-4ece-9cd8-41331cb83b01 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.794371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ece23f-faf2-4870-bece-2dbb1a7ac5b9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:17.800216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:17.805391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:17.836562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:17.837475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.838063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.838480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.839480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:17.841076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:17.852546 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:17.853549 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:17.854455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:17.855179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acb8d0-159d-4ece-9cd8-41331cb83b01 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:17.857944 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 428/851] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:17.864713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.895804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:17.896468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:17.896860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:17.897257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:17.912653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:17.914161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:17.932275 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:17.933272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:17.934930 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:17.934930 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:17.934930 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:17.934930 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 424/852] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:17.939092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:17.989064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90b4eb-48c4-4b8d-a166-306cb46db80f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.026949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.027894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.028339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.028761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.029545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:18.029873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.031232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.041363 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 429/853] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:18.041363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:18.054756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfafa01-3aa8-484f-870b-63176763a326 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.062597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:18.078331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:18.093385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.094028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.094312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.094831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.095221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:18.095524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.096942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.108789 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 425/854] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:18.109414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:18.155529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143b83-a901-4369-91d7-aedd41d2cdbb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.196907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a7fe-1ebe-442f-af4f-4d4ba207c3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.197545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a7fe-1ebe-442f-af4f-4d4ba207c3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a7fe-1ebe-442f-af4f-4d4ba207c3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.198126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672a7fe-1ebe-442f-af4f-4d4ba207c3f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.211148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672a7fe-1ebe-442f-af4f-4d4ba207c3f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2672a7fe-1ebe-442f-af4f-4d4ba207c3f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:18.218210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:18.219192 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:18.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 430/855] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:18.221712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a7fe-1ebe-442f-af4f-4d4ba207c3f0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:18.232707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6fc99-4170-4deb-a732-88a4e5582302 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.270277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.270886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.271162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.271431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.272059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:18.272415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.273824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.279203 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 426/856] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:18.280052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:18.318253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:18.367637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:18.377256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672a7fe-1ebe-442f-af4f-4d4ba207c3f0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.426444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0160817-99b2-48ed-8d02-56c0a5293157 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.438832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.440515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.441057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.442177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.453316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.455876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:18.477267 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:18.478520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:18.479602 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:18.479602 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:18.479602 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:18.479602 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:18.479602 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:18.479602 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:18.479602 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:18.479602 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:18.481864 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 431/857] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:18.482781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.483714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.484141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.484547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.485428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:18.485930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.487824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.489038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:18.501295 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 427/858] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:18.508731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:18.643953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa022db-5615-463f-ad92-27e1c561e425 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.644910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:18.658859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5daea-7136-4ec3-8d15-b2311b447729 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.688907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.689920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.690451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.690918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.691799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:18.692428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.694775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.704694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.706782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.706899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.707272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.721776 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 432/859] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:18.723701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:18.724405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.726013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:18.746572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:18.748542 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:18.749656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:18.749656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:18.749656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:18.749656 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 428/860] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:18.750252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:18.756131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:18.856799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae59f-8bcb-4a4f-bf52-99c90f638061 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.897222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6975c3-36cd-4336-ba2e-14e20a889d62 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:18.898661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.901554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.902069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.902544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.903533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=60761ed7c27f4018a6b4791b3b060bb0) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:18.904036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.905780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.918235 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 433/861] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/role_assignments?user.id=60761ed7c27f4018a6b4791b3b060bb0&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:18.941159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:18.941971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:18.942268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:18.942618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:18.943301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:18.943614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:18.945048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:18.960537 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 429/862] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:18.966511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:19.075240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:19.103873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405581-3470-4043-a5e8-f76ae9ef08e1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:19.145437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:19.146414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:19.146895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:19.147416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:19.157846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:19.160268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:19.176051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:19.177237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:19.178223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:19.179394 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 434/863] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:23:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:19.184888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:19.191560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:19.310970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4c6e1-07a8-478d-85dd-e7a331323f3d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:19.353774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:19.354391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:19.354670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:19.354942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:19.355694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:19.356004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:19.357637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:19.381529 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 430/864] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 30 09:23:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:19.389706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:19.503918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d87aa-748a-454e-9862-36ee146186cd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:19.543101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f561a-49bb-495e-b9a1-7cdf4920b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:19.543712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f561a-49bb-495e-b9a1-7cdf4920b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:19.543997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f561a-49bb-495e-b9a1-7cdf4920b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:19.544273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f561a-49bb-495e-b9a1-7cdf4920b1f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:19.555824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:19.616593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f561a-49bb-495e-b9a1-7cdf4920b1f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:19.618192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f561a-49bb-495e-b9a1-7cdf4920b1f0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:19.733100 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 435/865] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:23:19.738721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:19.803333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:19.847538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b1526-35e8-4912-b25c-30c83927a86b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:19.884973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83186be-2ddf-4067-8847-e5ee984f2a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:19.885608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83186be-2ddf-4067-8847-e5ee984f2a6c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:19.885916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83186be-2ddf-4067-8847-e5ee984f2a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:19.886192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83186be-2ddf-4067-8847-e5ee984f2a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:19.896823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83186be-2ddf-4067-8847-e5ee984f2a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:19.898320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83186be-2ddf-4067-8847-e5ee984f2a6c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f83186be-2ddf-4067-8847-e5ee984f2a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:19.914017 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:19.914993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:19.915887 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:19.915887 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:19.915887 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:19.915887 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 431/866] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:19.920955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f561a-49bb-495e-b9a1-7cdf4920b1f0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:19.960186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb8c06-5f7a-4d64-a0df-c3b1e3e7c597 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:19.960556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb8c06-5f7a-4d64-a0df-c3b1e3e7c597 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:19.960730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb8c06-5f7a-4d64-a0df-c3b1e3e7c597 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:19.962583 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 432/867] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:19.968520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291ade0-e125-4057-a8dd-badd3fdc7bad None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:19.968800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291ade0-e125-4057-a8dd-badd3fdc7bad None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:19.969073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291ade0-e125-4057-a8dd-badd3fdc7bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:19.984186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:19.992003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b291ade0-e125-4057-a8dd-badd3fdc7bad None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:19.993560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b291ade0-e125-4057-a8dd-badd3fdc7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:20.045105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f561a-49bb-495e-b9a1-7cdf4920b1f0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:20.069913 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 433/868] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:20.088483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:20.089777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:20.090308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:20.090787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:20.091627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:20.092152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:20.093747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:20.106668 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 436/869] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:20.113092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291ade0-e125-4057-a8dd-badd3fdc7bad None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:20.162472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:20.220242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291ade0-e125-4057-a8dd-badd3fdc7bad None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:20.256525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:20.257504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:20.258024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:20.258517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:20.265878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:20.268142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:20.280966 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:20.281913 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:20.282789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:20.282789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:20.282789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:20.282789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:20.282789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:20.282789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:20.282789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:20.282789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:20.289967 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 434/870] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:20.291916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:20.348657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:20.432126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:20.435491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ebb1e-3a5b-4977-b084-f6e51ee568a2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:20.472045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:20.472942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:20.473493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:20.474027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:20.474968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:20.475416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:20.477030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:20.477686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fb6dc-8395-478a-992f-256c41ea784d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:20.480823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:20.494259 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 437/871] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:20.500642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:20.527336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49c324-f374-4c3c-b20b-4b7a1d90e909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:20.528367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49c324-f374-4c3c-b20b-4b7a1d90e909 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:20.528864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49c324-f374-4c3c-b20b-4b7a1d90e909 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:20.529359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49c324-f374-4c3c-b20b-4b7a1d90e909 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:20.530349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49c324-f374-4c3c-b20b-4b7a1d90e909 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:20.532094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49c324-f374-4c3c-b20b-4b7a1d90e909 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:20.555168 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 435/872] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:23:20 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:23:20.636714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278c2d-e455-4dae-9d92-ef9a468bfeba None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:20.684501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:20.685501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:20.686021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:20.686587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:20.687600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=29179e1dc1b045a08cbd7feadc2ececd) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:20.688133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:20.690744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:20.693774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b369d-36f3-4be9-9c45-c1e197ce97ca None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:20.694058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b369d-36f3-4be9-9c45-c1e197ce97ca None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:20.694326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b369d-36f3-4be9-9c45-c1e197ce97ca None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:20.695959 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 436/873] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:20.699336 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 438/874] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:20 2022] GET /identity/v3/role_assignments?user.id=29179e1dc1b045a08cbd7feadc2ececd&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:20.701652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a7fed-d6ca-4ba6-85f9-689d589dae5f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:20.701959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a7fed-d6ca-4ba6-85f9-689d589dae5f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:20.702220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a7fed-d6ca-4ba6-85f9-689d589dae5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:20.729837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fc3a7fed-d6ca-4ba6-85f9-689d589dae5f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:20.731548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fc3a7fed-d6ca-4ba6-85f9-689d589dae5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:20.812799 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 437/875] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:21.086097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:21.112519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:21.204253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fd153-14cf-411f-997c-761d942a7aff None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:21.240844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867d798-e4ba-4a34-bd0d-7e1d36e1dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:21.241454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867d798-e4ba-4a34-bd0d-7e1d36e1dded None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:21.241762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867d798-e4ba-4a34-bd0d-7e1d36e1dded None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:21.242035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867d798-e4ba-4a34-bd0d-7e1d36e1dded None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:21.242699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867d798-e4ba-4a34-bd0d-7e1d36e1dded None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:21.244063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867d798-e4ba-4a34-bd0d-7e1d36e1dded None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:21.254919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61a13-16b5-4265-832f-dd3233bee861 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:21.255416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61a13-16b5-4265-832f-dd3233bee861 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:21.255493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61a13-16b5-4265-832f-dd3233bee861 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:21.258357 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 438/876] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:21.262288 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 439/877] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:23:21 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:21.262489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11cfef-84db-41c5-ba70-fddd337f108e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:21.262720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11cfef-84db-41c5-ba70-fddd337f108e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:21.262998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11cfef-84db-41c5-ba70-fddd337f108e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:21.288649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ed11cfef-84db-41c5-ba70-fddd337f108e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:21.290047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ed11cfef-84db-41c5-ba70-fddd337f108e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:21.378129 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 439/878] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:21.651439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867d798-e4ba-4a34-bd0d-7e1d36e1dded None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:21.737887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:21.758433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867d798-e4ba-4a34-bd0d-7e1d36e1dded None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:21.798265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:21.799061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:21.799422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:21.799697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:21.800280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:21.801655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:21.810404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:21.811392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:21.812222 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 440/879] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:21.818775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11cfef-84db-41c5-ba70-fddd337f108e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:21.840789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:21.928389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11cfef-84db-41c5-ba70-fddd337f108e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:21.963245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54102612-62ad-48db-aa97-177eccc74a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:21.963861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102612-62ad-48db-aa97-177eccc74a55 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:21.964144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102612-62ad-48db-aa97-177eccc74a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:21.964418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102612-62ad-48db-aa97-177eccc74a55 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:21.965055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54102612-62ad-48db-aa97-177eccc74a55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:21.965358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54102612-62ad-48db-aa97-177eccc74a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:21.966776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54102612-62ad-48db-aa97-177eccc74a55 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:21.972585 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 440/880] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:21.978426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:22.086866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f6071-dedf-433e-9cd0-d82a69e20297 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:22.122938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8ffdb-8d27-43e8-8a8c-7ef45c64ad37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:22.123556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8ffdb-8d27-43e8-8a8c-7ef45c64ad37 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:22.123811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8ffdb-8d27-43e8-8a8c-7ef45c64ad37 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:22.124082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8ffdb-8d27-43e8-8a8c-7ef45c64ad37 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:22.124706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8ffdb-8d27-43e8-8a8c-7ef45c64ad37 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:22.125005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8ffdb-8d27-43e8-8a8c-7ef45c64ad37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:22.126391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8ffdb-8d27-43e8-8a8c-7ef45c64ad37 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:22.137597 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 441/881] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:22.959071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1cd36-726d-471f-88dc-38a316a56d1d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:22.960061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1cd36-726d-471f-88dc-38a316a56d1d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:22.960556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1cd36-726d-471f-88dc-38a316a56d1d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:22.962389 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 441/882] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:22.967872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8c4da-a053-41c2-b618-1f46b3d51273 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:22.968395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8c4da-a053-41c2-b618-1f46b3d51273 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:22.968876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8c4da-a053-41c2-b618-1f46b3d51273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:22.987618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2ef8c4da-a053-41c2-b618-1f46b3d51273 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:22.989206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2ef8c4da-a053-41c2-b618-1f46b3d51273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:22.991551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:23.038240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef88910-2f06-47aa-b553-777b4cb2a3fc None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.038554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef88910-2f06-47aa-b553-777b4cb2a3fc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.038789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef88910-2f06-47aa-b553-777b4cb2a3fc None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.040369 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 442/883] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:23:23.048353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcec6-2e50-4ee9-a34a-2d3537432fdd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.048353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcec6-2e50-4ee9-a34a-2d3537432fdd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.048353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcec6-2e50-4ee9-a34a-2d3537432fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.065770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9adbcec6-2e50-4ee9-a34a-2d3537432fdd None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:23.067006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9adbcec6-2e50-4ee9-a34a-2d3537432fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:23.083589 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 442/884] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:23.137392 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 443/885] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:23.359290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8c4da-a053-41c2-b618-1f46b3d51273 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:23.413813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbcec6-2e50-4ee9-a34a-2d3537432fdd None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:23.431214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:23.489535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8c4da-a053-41c2-b618-1f46b3d51273 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:23.523635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbcec6-2e50-4ee9-a34a-2d3537432fdd None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:23.528211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b426ccc-496c-4ef2-ac3f-021fb8dfaa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:23.528845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426ccc-496c-4ef2-ac3f-021fb8dfaa91 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.529121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426ccc-496c-4ef2-ac3f-021fb8dfaa91 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.529389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426ccc-496c-4ef2-ac3f-021fb8dfaa91 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.529993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b426ccc-496c-4ef2-ac3f-021fb8dfaa91 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:23.531362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b426ccc-496c-4ef2-ac3f-021fb8dfaa91 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2b426ccc-496c-4ef2-ac3f-021fb8dfaa91 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:23.542256 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:23.543233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:23.544084 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 443/886] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:23.545367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b06094-d697-4eaa-b074-8267e37fc79f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.545367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b06094-d697-4eaa-b074-8267e37fc79f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.545536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b06094-d697-4eaa-b074-8267e37fc79f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.549690 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 444/887] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:23.549690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b06094-d697-4eaa-b074-8267e37fc79f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:23.552520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:23.568386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fce0c0-4e0c-4472-84b7-44c257d59e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:23.569318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fce0c0-4e0c-4472-84b7-44c257d59e36 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.569793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fce0c0-4e0c-4472-84b7-44c257d59e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.570278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fce0c0-4e0c-4472-84b7-44c257d59e36 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.581760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fce0c0-4e0c-4472-84b7-44c257d59e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-73fce0c0-4e0c-4472-84b7-44c257d59e36 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:23.586703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:23.587875 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:23.588862 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:23.591013 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 444/888] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 30 09:23:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:23.592468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4c70-f1eb-49e0-a710-ebb3ebd4e3da None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.592883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4c70-f1eb-49e0-a710-ebb3ebd4e3da None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.593302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4c70-f1eb-49e0-a710-ebb3ebd4e3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.614059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-887c4c70-f1eb-49e0-a710-ebb3ebd4e3da None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:23.615176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-887c4c70-f1eb-49e0-a710-ebb3ebd4e3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:23.662700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b06094-d697-4eaa-b074-8267e37fc79f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:23.687917 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 445/889] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:23.688638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c4c70-f1eb-49e0-a710-ebb3ebd4e3da None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:23.692667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:23.705506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:23.706485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.706973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.707451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.708344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:23.709014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:23.710925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:23.722163 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 445/890] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:23.723860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:23.798285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c4c70-f1eb-49e0-a710-ebb3ebd4e3da None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:23.800370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:23.834804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:23.835410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.835686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.835956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.836617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:23.836914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:23.838485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:23.841350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd39bcb-2ecb-441a-8b32-a35d94b74c15 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:23.852132 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 446/891] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 30 09:23:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:23.853017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:23.882355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:23.883755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:23.884329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:23.884811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:23.885651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:23.886163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:23.887731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:23.899258 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 446/892] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:23.900946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:23.928326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:23.966632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b130d3-e67c-4117-9287-44c82beb9301 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.007169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.008102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.008572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.008839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.017163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8643f-431f-4c37-82f1-b53a40411fa7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.019816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.021279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:24.037320 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:24.038325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:24.044387 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:24.044387 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:24.044387 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:24.044387 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 447/893] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:24.044387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.053126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:24.059760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.060365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.060640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.060907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.061736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] RBAC: Adding query filter params (interface=public, service_id=9c8e6401a98e4177aab15208ef7630d6, region_id=RegionOne) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:24.062423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.063418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.071593 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 447/894] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 30 09:23:23 2022] GET /identity/v3/endpoints?service_id=9c8e6401a98e4177aab15208ef7630d6&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:24.072489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.151443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985fc38-aa3a-4cbf-9d9e-a3b68de7191c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.170530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:24.176669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7e946-d3dc-4e4a-8aa6-89212989360d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.188655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.189337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.189741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.190020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.190605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.192048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:24.201470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:24.202431 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:24.203968 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 448/895] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:24.204761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.217759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.218338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.218620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.218897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.219537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:24.219841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.221855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.234074 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 448/896] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:24.239382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.299853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:24.317429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091a3d-bc76-4986-841d-156f6c78e30c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.349929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a3e4d-082c-4be7-b6fe-740128883fec None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.352234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8bb37-a9ef-4bdc-b2a8-dc36407d68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.352832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8bb37-a9ef-4bdc-b2a8-dc36407d68b3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.353112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8bb37-a9ef-4bdc-b2a8-dc36407d68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.353380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8bb37-a9ef-4bdc-b2a8-dc36407d68b3 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.354005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8bb37-a9ef-4bdc-b2a8-dc36407d68b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.355372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8bb37-a9ef-4bdc-b2a8-dc36407d68b3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.369664 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 449/897] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:23:24.369664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8bb37-a9ef-4bdc-b2a8-dc36407d68b3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.410657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2a4d1-1823-4d28-a045-08c66338d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.411316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a4d1-1823-4d28-a045-08c66338d569 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.411696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a4d1-1823-4d28-a045-08c66338d569 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.411969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a4d1-1823-4d28-a045-08c66338d569 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.412742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a4d1-1823-4d28-a045-08c66338d569 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:24.413050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a4d1-1823-4d28-a045-08c66338d569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.414498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a4d1-1823-4d28-a045-08c66338d569 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.419856 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 449/898] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:24.423369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62e1db-3d8c-4f53-82f5-e6f7fd0bd951 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.423828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62e1db-3d8c-4f53-82f5-e6f7fd0bd951 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.424195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62e1db-3d8c-4f53-82f5-e6f7fd0bd951 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.429676 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 450/899] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:24.429947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:24.430052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62e1db-3d8c-4f53-82f5-e6f7fd0bd951 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.492951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8bb37-a9ef-4bdc-b2a8-dc36407d68b3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.530687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b0c1e-3b22-4351-87ac-b8fe830e8bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.531706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0c1e-3b22-4351-87ac-b8fe830e8bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.532296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0c1e-3b22-4351-87ac-b8fe830e8bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.532575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b0c1e-3b22-4351-87ac-b8fe830e8bf6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.542179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b0c1e-3b22-4351-87ac-b8fe830e8bf6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.542371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62e1db-3d8c-4f53-82f5-e6f7fd0bd951 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.543686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b0c1e-3b22-4351-87ac-b8fe830e8bf6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-981b0c1e-3b22-4351-87ac-b8fe830e8bf6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:24.555737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:24.556704 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:24.557622 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 450/900] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:24.558256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2cd3c-9f0c-4c4a-b500-f870b337fdf8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.558325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2cd3c-9f0c-4c4a-b500-f870b337fdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.558550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2cd3c-9f0c-4c4a-b500-f870b337fdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.566395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:24.578697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b2e2cd3c-9f0c-4c4a-b500-f870b337fdf8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:24.579980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b2e2cd3c-9f0c-4c4a-b500-f870b337fdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:24.584220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.584849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.585131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.585405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.586110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:24.586415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.587782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.594069 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 451/901] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:24.594713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.657742 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 451/902] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:24.657742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2cd3c-9f0c-4c4a-b500-f870b337fdf8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.694091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:24.714569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847cd02-1dd0-4c6a-82fe-72d28f66d011 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.758984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.759698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.759986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.760361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.761107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:24.761414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.762873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.763069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2cd3c-9f0c-4c4a-b500-f870b337fdf8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.783393 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 452/903] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:24.788504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.807008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.808022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.808313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.808590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.809311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] RBAC: Adding query filter params (interface=public, service_id=164e8edb491a4b6c8caf27b5997b8fcd, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:24.809652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.811582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.823903 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 452/904] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/endpoints?service_id=164e8edb491a4b6c8caf27b5997b8fcd&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:24.825209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:24.837337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:24.908173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6befc-d20b-49fb-b829-24f3f1662240 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.941880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:24.945839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d7213-dd77-4881-a9b1-4ac458d992e3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:24.949907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391180ce-8271-4841-b652-a899809ed3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.950522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391180ce-8271-4841-b652-a899809ed3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.950802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391180ce-8271-4841-b652-a899809ed3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.951074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391180ce-8271-4841-b652-a899809ed3a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.984940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee460-5be6-4c97-8f69-b4d585d67376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:24.985599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee460-5be6-4c97-8f69-b4d585d67376 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:24.985880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee460-5be6-4c97-8f69-b4d585d67376 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:24.986159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee460-5be6-4c97-8f69-b4d585d67376 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:24.986873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edee460-5be6-4c97-8f69-b4d585d67376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:24.988324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edee460-5be6-4c97-8f69-b4d585d67376 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:24.993803 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 453/905] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:23:24.994557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee460-5be6-4c97-8f69-b4d585d67376 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:25.038422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391180ce-8271-4841-b652-a899809ed3a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:25.039893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391180ce-8271-4841-b652-a899809ed3a8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:25.086537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:25.139352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee460-5be6-4c97-8f69-b4d585d67376 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:25.172731 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 453/906] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:24 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:23:25.178050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391180ce-8271-4841-b652-a899809ed3a8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:25.184897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e94ad-e274-4e98-b225-1e2c62748611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:25.185744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e94ad-e274-4e98-b225-1e2c62748611 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:25.186184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e94ad-e274-4e98-b225-1e2c62748611 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:25.186597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e94ad-e274-4e98-b225-1e2c62748611 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:25.187608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e94ad-e274-4e98-b225-1e2c62748611 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:25.189855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e94ad-e274-4e98-b225-1e2c62748611 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:25.208538 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 454/907] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:23:24 2022] POST /identity/v3/services => generated 241 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:25.296824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391180ce-8271-4841-b652-a899809ed3a8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:25.344079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:25.344735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:25.345015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:25.345288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:25.349517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:25.358624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:25.360235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:25.377341 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:25.378363 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:25.379229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:25.379229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:25.379229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:25.379229 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 454/908] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:25.384640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e94ad-e274-4e98-b225-1e2c62748611 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:25.424673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:25.510637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e94ad-e274-4e98-b225-1e2c62748611 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:25.546587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:25.547581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:25.548072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:25.549046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:25.550116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:25.550619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:25.552258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:25.565218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 455/909] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:25.570548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:25.676548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b785c-dba6-4e63-8b6f-fc02a90f52be None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:25.717051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:25.717722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:25.718046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:25.718326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:25.726525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:25.727967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:25.739641 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:25.740533 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:25.741483 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 455/910] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:25.750735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:25.840398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:25.857724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4461f8-0e9d-459a-b576-91d4c800bc4a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:25.892787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24e196-0c72-4ae7-a5e6-5ff395b12318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:25.893769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24e196-0c72-4ae7-a5e6-5ff395b12318 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:25.894280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24e196-0c72-4ae7-a5e6-5ff395b12318 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:25.894745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24e196-0c72-4ae7-a5e6-5ff395b12318 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:25.895641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24e196-0c72-4ae7-a5e6-5ff395b12318 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:25.896140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24e196-0c72-4ae7-a5e6-5ff395b12318 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:25.897798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24e196-0c72-4ae7-a5e6-5ff395b12318 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:25.910853 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 456/911] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:25.917654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:25.982115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:26.026595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af625b11-e870-41b2-8a76-7a39b31810ee None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:26.061796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a71041-b956-4e45-99e9-7c3833029393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:26.062408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71041-b956-4e45-99e9-7c3833029393 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:26.062687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71041-b956-4e45-99e9-7c3833029393 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:26.062963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a71041-b956-4e45-99e9-7c3833029393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:26.063688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a71041-b956-4e45-99e9-7c3833029393 None admin] RBAC: Adding query filter params (role.id=61285fdf54ab4933b5e0b2f767155362, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=29179e1dc1b045a08cbd7feadc2ececd) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:26.063995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a71041-b956-4e45-99e9-7c3833029393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:26.066187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a71041-b956-4e45-99e9-7c3833029393 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:26.072190 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 456/912] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:25 2022] GET /identity/v3/role_assignments?user.id=29179e1dc1b045a08cbd7feadc2ececd&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=61285fdf54ab4933b5e0b2f767155362 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:26.603927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d3f3-2e2e-4300-87c2-f535a5b7d9c9 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:26.604442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d3f3-2e2e-4300-87c2-f535a5b7d9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:26.604518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d3f3-2e2e-4300-87c2-f535a5b7d9c9 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:26.605974 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 457/913] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:26.612266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884f1d4-41fa-4a9c-a12f-ae91179e2d9d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:26.613090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884f1d4-41fa-4a9c-a12f-ae91179e2d9d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:26.613571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884f1d4-41fa-4a9c-a12f-ae91179e2d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:26.632253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8884f1d4-41fa-4a9c-a12f-ae91179e2d9d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:26.633496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8884f1d4-41fa-4a9c-a12f-ae91179e2d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:26.703118 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 457/914] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:26.974242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d3f3-2e2e-4300-87c2-f535a5b7d9c9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:27.092222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d3f3-2e2e-4300-87c2-f535a5b7d9c9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:27.128531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:27.129168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.129449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.129766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.130354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:27.131738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:27.141044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:27.142058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:27.143512 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 458/915] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:27.150549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884f1d4-41fa-4a9c-a12f-ae91179e2d9d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:27.202119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:27.259289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884f1d4-41fa-4a9c-a12f-ae91179e2d9d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:27.293945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d373c8-deac-45c2-92c7-eaa77cd4621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:27.294560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d373c8-deac-45c2-92c7-eaa77cd4621e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.294839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d373c8-deac-45c2-92c7-eaa77cd4621e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.295110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d373c8-deac-45c2-92c7-eaa77cd4621e None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.295743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d373c8-deac-45c2-92c7-eaa77cd4621e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:27.296044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d373c8-deac-45c2-92c7-eaa77cd4621e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:27.297425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d373c8-deac-45c2-92c7-eaa77cd4621e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:27.303563 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 458/916] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:27.309245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:27.321796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:27.330529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81ff9d-f3af-4fd0-80ed-15ba8e225785 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.330839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81ff9d-f3af-4fd0-80ed-15ba8e225785 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.331121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81ff9d-f3af-4fd0-80ed-15ba8e225785 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.336240 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 459/917] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:23:27.337979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b326ba-0326-4560-b295-d74a623025c7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.338409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b326ba-0326-4560-b295-d74a623025c7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.338837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b326ba-0326-4560-b295-d74a623025c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.358464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-92b326ba-0326-4560-b295-d74a623025c7 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:27.360001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-92b326ba-0326-4560-b295-d74a623025c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:27.439788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b32836-0c8e-41ba-85e9-83f7822d1e52 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:27.449048 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 460/918] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:27.478625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9adb7f-8f69-4b0d-9080-9a74f375ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:27.479261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9adb7f-8f69-4b0d-9080-9a74f375ff8a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.479545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9adb7f-8f69-4b0d-9080-9a74f375ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.479820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9adb7f-8f69-4b0d-9080-9a74f375ff8a None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.480475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9adb7f-8f69-4b0d-9080-9a74f375ff8a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:27.480784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9adb7f-8f69-4b0d-9080-9a74f375ff8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:27.482247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9adb7f-8f69-4b0d-9080-9a74f375ff8a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:27.489298 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 459/919] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 30 09:23:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:27.494935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b326ba-0326-4560-b295-d74a623025c7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:27.530090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:27.583643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d9dca-25e1-43a9-8d81-60f6d023ad53 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.583963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d9dca-25e1-43a9-8d81-60f6d023ad53 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.584201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d9dca-25e1-43a9-8d81-60f6d023ad53 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.585827 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 460/920] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:23:27.590703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dea9bd-8333-40ee-8fbf-4e5eb04eeb48 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.590989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dea9bd-8333-40ee-8fbf-4e5eb04eeb48 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.591266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dea9bd-8333-40ee-8fbf-4e5eb04eeb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.601736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b326ba-0326-4560-b295-d74a623025c7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:27.609894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-22dea9bd-8333-40ee-8fbf-4e5eb04eeb48 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:27.611193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-22dea9bd-8333-40ee-8fbf-4e5eb04eeb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:27.643371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4f881-bb66-47f9-8b25-41446a47742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:27.644030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f881-bb66-47f9-8b25-41446a47742b None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.644254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f881-bb66-47f9-8b25-41446a47742b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.644523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f881-bb66-47f9-8b25-41446a47742b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.645205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4f881-bb66-47f9-8b25-41446a47742b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:27.646592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4f881-bb66-47f9-8b25-41446a47742b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:27.674148 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 461/921] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:23:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:27.703877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:27.707061 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 461/922] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:27.735862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4f881-bb66-47f9-8b25-41446a47742b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:27.847056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:27.857393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4f881-bb66-47f9-8b25-41446a47742b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:27.894369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:27.895005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:27.895283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:27.895554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:27.896146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:27.897518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:27.907640 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:27.908583 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:27.910392 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 462/923] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 30 09:23:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:27.914765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dea9bd-8333-40ee-8fbf-4e5eb04eeb48 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:27.993206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:28.036156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dea9bd-8333-40ee-8fbf-4e5eb04eeb48 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:28.081669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:28.082334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.082664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.082947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.083582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:28.083934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:28.085328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:28.091679 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 462/924] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:28.097663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:28.128732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df5c96-66cc-43ff-bcba-f1cae5217beb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:28.164283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:28.179241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424f05-e03b-4381-830a-b687daec068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:28.179889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424f05-e03b-4381-830a-b687daec068c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.180172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424f05-e03b-4381-830a-b687daec068c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.180449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89424f05-e03b-4381-830a-b687daec068c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.181038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89424f05-e03b-4381-830a-b687daec068c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:28.182978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89424f05-e03b-4381-830a-b687daec068c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-89424f05-e03b-4381-830a-b687daec068c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:28.195461 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:28.196558 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:28.197851 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 463/925] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:28.203470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424f05-e03b-4381-830a-b687daec068c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:28.214100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54259bc-c24a-4b7f-b629-b7b498e20310 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:28.235224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:28.253389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:28.255171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.255523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.255864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.256579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:28.256961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:28.258522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:28.269855 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 463/926] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:28.272180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:28.346092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89424f05-e03b-4381-830a-b687daec068c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:28.385904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b1470-2b3f-4978-ac47-1e1ad494564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:28.386523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b1470-2b3f-4978-ac47-1e1ad494564c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.386854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b1470-2b3f-4978-ac47-1e1ad494564c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.387345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b1470-2b3f-4978-ac47-1e1ad494564c None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.387774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1f641-8458-4ed4-9ee7-42ed19ab7ce8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:28.388442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b1470-2b3f-4978-ac47-1e1ad494564c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:28.388966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b1470-2b3f-4978-ac47-1e1ad494564c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:28.391251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b1470-2b3f-4978-ac47-1e1ad494564c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:28.396635 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 464/927] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:28.397988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327321-0898-4810-8dce-05966448ec51 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.398282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327321-0898-4810-8dce-05966448ec51 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.398555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327321-0898-4810-8dce-05966448ec51 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.400165 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 465/928] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:28.405495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327321-0898-4810-8dce-05966448ec51 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:28.420175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:28.431140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcea1a2-cfc1-49b4-abea-2dfa28bd475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:28.431751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcea1a2-cfc1-49b4-abea-2dfa28bd475b None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.432034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcea1a2-cfc1-49b4-abea-2dfa28bd475b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.432315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcea1a2-cfc1-49b4-abea-2dfa28bd475b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.432981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcea1a2-cfc1-49b4-abea-2dfa28bd475b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:28.434399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcea1a2-cfc1-49b4-abea-2dfa28bd475b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:28.456164 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 464/929] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:23:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:23:28.457427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc1e7c-136b-4725-9254-439cda9aab1f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.458116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc1e7c-136b-4725-9254-439cda9aab1f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.458397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc1e7c-136b-4725-9254-439cda9aab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.481995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-edcc1e7c-136b-4725-9254-439cda9aab1f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:28.483710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-edcc1e7c-136b-4725-9254-439cda9aab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:28.512909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:28.547895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327321-0898-4810-8dce-05966448ec51 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:28.582075 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 465/930] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:28.588539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:28.589330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.589709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.590059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.590832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:28.591213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:28.592791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:28.599621 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 466/931] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:28.605405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc1e7c-136b-4725-9254-439cda9aab1f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:28.724274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc1e7c-136b-4725-9254-439cda9aab1f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:28.739177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:28.762945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:28.763563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.763843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.764115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.764813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] RBAC: Adding query filter params (interface=public, service_id=dcf7d8cdf2e941b2875648c9d41b1898, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:28.765122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:28.766534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:28.772743 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 466/932] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 30 09:23:28 2022] GET /identity/v3/endpoints?service_id=dcf7d8cdf2e941b2875648c9d41b1898&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:28.778656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:28.868404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:28.891125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031695-2aa4-4f2a-8ca3-6ed757f35034 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:28.928464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d833-26ac-4b0d-ae56-62c24edce4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:28.929455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d833-26ac-4b0d-ae56-62c24edce4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:28.929965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d833-26ac-4b0d-ae56-62c24edce4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:28.930529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d833-26ac-4b0d-ae56-62c24edce4b7 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:28.931421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619d833-26ac-4b0d-ae56-62c24edce4b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:28.932992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619d833-26ac-4b0d-ae56-62c24edce4b7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:28.940886 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 467/933] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:23:28 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:23:28.989090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5b0bc-19eb-4b42-a817-1e40782ae7e2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:29.028409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33041a2b-a619-430b-9391-b0ae54e4b6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:29.029068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33041a2b-a619-430b-9391-b0ae54e4b6cb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:29.029349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33041a2b-a619-430b-9391-b0ae54e4b6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:29.029643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33041a2b-a619-430b-9391-b0ae54e4b6cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:29.042248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33041a2b-a619-430b-9391-b0ae54e4b6cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-33041a2b-a619-430b-9391-b0ae54e4b6cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:29.047817 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:29.048963 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:29.049802 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 467/934] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 30 09:23:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:29.057084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d833-26ac-4b0d-ae56-62c24edce4b7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:29.069972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:29.168971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619d833-26ac-4b0d-ae56-62c24edce4b7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:29.204066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:29.204766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:29.205614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:29.205919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:29.206642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:29.207013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:29.208383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:29.216946 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 468/935] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 30 09:23:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:29.226385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33041a2b-a619-430b-9391-b0ae54e4b6cb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:29.264623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:29.340132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33041a2b-a619-430b-9391-b0ae54e4b6cb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:29.381330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:29.382000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:29.382281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:29.382550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:29.400847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:29.407348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:29.422523 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:29.424093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:29.425074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:29.425074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:29.425074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:29.425074 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 468/936] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:29.431763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:29.510003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:29.544504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268ea87-ed61-41b7-9ac8-18ac72a43cef None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:29.580106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:29.580773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:29.581062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:29.581335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:29.582008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:29.582310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:29.583673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:29.596477 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 469/937] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:29.602015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:29.699668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:29.715508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b19b-c9a9-4332-80c3-aa329a404697 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:29.754268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:29.755728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:29.756232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:29.756729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:29.764733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:29.766458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:29.778911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:29.779894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:29.780858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:29.784046 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 469/938] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:23:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:29.790144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:29.899756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6fafb-8b95-4a7f-9418-7f98e15c73bc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:29.936041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:29.944528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a9603-f807-4a76-9f03-4ba0d9c11090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:29.945388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a9603-f807-4a76-9f03-4ba0d9c11090 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:29.945895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a9603-f807-4a76-9f03-4ba0d9c11090 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:29.946367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a9603-f807-4a76-9f03-4ba0d9c11090 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:29.947287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a9603-f807-4a76-9f03-4ba0d9c11090 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:29.947812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a9603-f807-4a76-9f03-4ba0d9c11090 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:29.949438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a9603-f807-4a76-9f03-4ba0d9c11090 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:29.968415 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 470/939] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 30 09:23:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:29.974892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:30.034528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d755da4-e5fa-48c6-80ae-e5ae515630d0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.035182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d755da4-e5fa-48c6-80ae-e5ae515630d0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.035650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d755da4-e5fa-48c6-80ae-e5ae515630d0 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.038236 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 471/940] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:23:30.044696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2571-8266-4bc8-a7f6-9f67c94b22f6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.044696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2571-8266-4bc8-a7f6-9f67c94b22f6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.044986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab2571-8266-4bc8-a7f6-9f67c94b22f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.063737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d7ab2571-8266-4bc8-a7f6-9f67c94b22f6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:30.065501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d7ab2571-8266-4bc8-a7f6-9f67c94b22f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:30.102391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac9dfe-15bf-439a-a82c-395023aa74d0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:30.138426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:30.139092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.139320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.139585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.150450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:30.152198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:30.155105 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 472/941] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 30 09:23:30.163084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:30.171901 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:30.172829 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:30.174568 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:30.174568 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:30.174568 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:30.176593 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 470/942] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:30.183045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab2571-8266-4bc8-a7f6-9f67c94b22f6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:30.289712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab2571-8266-4bc8-a7f6-9f67c94b22f6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:30.311269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:30.337585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:30.338559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.339076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.339555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.340406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:30.340911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:30.342514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:30.355701 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 473/943] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:30.362013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:30.451035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:30.483312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae1003-8177-4264-b454-b6a7700cba02 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:30.520639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:30.521270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.521546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.521841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.529254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:30.531018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:30.544525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:30.545426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:30.546341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:30.546341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:30.546341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:30.546341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:30.546341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:30.546341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:30.546341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:30.546341 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:30.549766 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 471/944] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:30.551994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:30.580076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76285fdf-c37f-43fa-9233-a05704bb246a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:30.617537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:30.621511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:30.622373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.622816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.623240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.624053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:30.625818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:30.630298 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING keystone.server.flask.common [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 30 09:23:30.657373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.sql.core [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 30 09:23:30.657373 dv-cinder-master-3-smb devstack@keystone.service[81372]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 30 09:23:30.657373 dv-cinder-master-3-smb devstack@keystone.service[81372]: [parameters: {'id': '9a7de1f994ce4cf08fe32b33c8f87d29', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 30 09:23:30.657373 dv-cinder-master-3-smb devstack@keystone.service[81372]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81372) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 30 09:23:30.666952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ffe04-e306-4164-af1b-bcce06d32f15 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a7de1f994ce4cf08fe32b33c8f87d29.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a7de1f994ce4cf08fe32b33c8f87d29. Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application conn.query(q) Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 09:23:30.676261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result.read() Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 30 09:23:30.677261 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application next(self.gen) Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application yield resource Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application next(self.gen) Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.session.rollback() Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.force_reraise() Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise self.value Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application session.commit() Sep 30 09:23:30.678480 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._prepare_impl() Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.session.flush() Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._flush(objects) Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application compat.raise_( Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application flush_context.execute() Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 30 09:23:30.679365 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rec.execute(self) Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 30 09:23:30.680514 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application conn.query(q) Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result.read() Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 09:23:30.681730 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [parameters: {'id': '9a7de1f994ce4cf08fe32b33c8f87d29', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:30.683034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:30.684090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a7de1f994ce4cf08fe32b33c8f87d29. Sep 30 09:23:30.685237 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:30.693098 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 474/945] 10.222.0.15 () {60 vars in 1140 bytes} [Fri Sep 30 09:23:30 2022] POST /identity/v3/projects => generated 262 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:30.695611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:30.710136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:30.710827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.711215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.711489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.712205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:30.712511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:30.714315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:30.734014 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 472/946] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:30.735425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:30.743426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:30.824789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe303-ef40-4162-b6a1-43b095d386a1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:30.849822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38493d4-a775-4d25-bd87-20240bc9d701 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:30.860524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc526c-7354-4056-8f4b-dc6bfa31589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:30.861757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc526c-7354-4056-8f4b-dc6bfa31589f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.862094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc526c-7354-4056-8f4b-dc6bfa31589f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.862402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc526c-7354-4056-8f4b-dc6bfa31589f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.869531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc526c-7354-4056-8f4b-dc6bfa31589f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:30.871232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc526c-7354-4056-8f4b-dc6bfa31589f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:30.871700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-eabc526c-7354-4056-8f4b-dc6bfa31589f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:30.883842 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:30.884755 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:30.885673 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 475/947] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:30.886709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c8f6-70b5-408d-9e00-f202a2c5061a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.886709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c8f6-70b5-408d-9e00-f202a2c5061a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.886866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c8f6-70b5-408d-9e00-f202a2c5061a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.891091 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 476/948] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:30.891608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72c8f6-70b5-408d-9e00-f202a2c5061a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:30.898800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ef940-7bbc-4040-b3b5-827db557b8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:30.899502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ef940-7bbc-4040-b3b5-827db557b8c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.899887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ef940-7bbc-4040-b3b5-827db557b8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.900177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ef940-7bbc-4040-b3b5-827db557b8c3 None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/29179e1dc1b045a08cbd7feadc2ececd/roles/61285fdf54ab4933b5e0b2f767155362` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.902803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ef940-7bbc-4040-b3b5-827db557b8c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=29179e1dc1b045a08cbd7feadc2ececd, role_id=61285fdf54ab4933b5e0b2f767155362)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:30.904265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ef940-7bbc-4040-b3b5-827db557b8c3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:30.921359 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 473/949] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:23:30 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/29179e1dc1b045a08cbd7feadc2ececd/roles/61285fdf54ab4933b5e0b2f767155362 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:23:30.922577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032027-eba5-495a-9d6c-5da56f69c74c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:30.922862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032027-eba5-495a-9d6c-5da56f69c74c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:30.923135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032027-eba5-495a-9d6c-5da56f69c74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:30.953555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0a032027-eba5-495a-9d6c-5da56f69c74c None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:30.955730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0a032027-eba5-495a-9d6c-5da56f69c74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:31.024549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:31.034271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72c8f6-70b5-408d-9e00-f202a2c5061a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:31.072017 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 474/950] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:31.086370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd0b2f-5f50-44a2-b09b-c847b2f86b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:31.087045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd0b2f-5f50-44a2-b09b-c847b2f86b99 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:31.087345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd0b2f-5f50-44a2-b09b-c847b2f86b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:31.087666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd0b2f-5f50-44a2-b09b-c847b2f86b99 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:31.088411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd0b2f-5f50-44a2-b09b-c847b2f86b99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:31.088719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd0b2f-5f50-44a2-b09b-c847b2f86b99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:31.090393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd0b2f-5f50-44a2-b09b-c847b2f86b99 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:31.111274 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 477/951] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:31.348045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d221d9-a18a-4420-b5d8-d74ac0914ffb None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:31.348386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d221d9-a18a-4420-b5d8-d74ac0914ffb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:31.348564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d221d9-a18a-4420-b5d8-d74ac0914ffb None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:31.350174 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 475/952] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:31.355656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927dec-c7b8-44e0-8b81-c2fbd0226f4d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:31.355945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927dec-c7b8-44e0-8b81-c2fbd0226f4d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:31.356220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927dec-c7b8-44e0-8b81-c2fbd0226f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:31.357743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d221d9-a18a-4420-b5d8-d74ac0914ffb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:31.377719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cb927dec-c7b8-44e0-8b81-c2fbd0226f4d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:31.379218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cb927dec-c7b8-44e0-8b81-c2fbd0226f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:31.456592 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 478/953] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:31.497442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d221d9-a18a-4420-b5d8-d74ac0914ffb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:31.510864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:31.537483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:31.538590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:31.539108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:31.539578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:31.540415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:31.542078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:31.552778 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:31.554008 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:31.554896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:31.557621 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 476/954] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:31.565672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb927dec-c7b8-44e0-8b81-c2fbd0226f4d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:31.687146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:31.714204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb927dec-c7b8-44e0-8b81-c2fbd0226f4d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:31.758147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:31.768838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:31.769462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:31.769768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:31.770041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:31.770682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:31.770982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:31.772343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:31.777924 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 479/955] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:31.788558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:31.888105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8195687-d3c1-4685-8852-0c7d966b60b0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:31.928720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:31.929351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:31.930492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:31.930766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:31.931353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:31.932740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:31.943280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:31.944990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:31.947174 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:31.953659 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 477/956] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:31.953659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:31.995027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d3619-6c6c-4144-9a00-3594d43e6c85 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:31.997024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:32.035756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:32.036522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:32.036905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:32.037279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:32.038040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:32.038614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:32.040076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:32.045006 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 480/957] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:32.079799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d107cc-a070-417e-8430-6de637a71314 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:32.119397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:32.120030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:32.120308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:32.120579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:32.121234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:32.121536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:32.122996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:32.128972 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 478/958] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:32.134945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:32.151486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:32.250591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82822d4-9672-496c-b7bf-40ecb63b56d1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:32.287315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21111465-382b-49c7-a1ee-88468e423a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:32.287955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21111465-382b-49c7-a1ee-88468e423a47 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:32.288239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21111465-382b-49c7-a1ee-88468e423a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:32.288525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21111465-382b-49c7-a1ee-88468e423a47 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:32.289175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21111465-382b-49c7-a1ee-88468e423a47 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:32.289482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21111465-382b-49c7-a1ee-88468e423a47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:32.290949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21111465-382b-49c7-a1ee-88468e423a47 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:32.297093 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 481/959] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:32.303370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:32.414618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54ef8e-7f36-4b81-94ff-554461942b3c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:32.416387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:32.451740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36515396-fdcc-448e-af60-a625a4877be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:32.452638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515396-fdcc-448e-af60-a625a4877be8 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:32.453066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515396-fdcc-448e-af60-a625a4877be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:32.453508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36515396-fdcc-448e-af60-a625a4877be8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:32.454383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36515396-fdcc-448e-af60-a625a4877be8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:32.456428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36515396-fdcc-448e-af60-a625a4877be8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:32.480377 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 479/960] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:23:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:33.240911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74638e3f-4f93-4a1d-9d19-cd621fd489b8 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:33.241464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74638e3f-4f93-4a1d-9d19-cd621fd489b8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:33.241464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74638e3f-4f93-4a1d-9d19-cd621fd489b8 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:33.243083 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 482/961] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:33.248556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101ec6-a41a-4310-bf3f-f02e44f0b669 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:33.248961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101ec6-a41a-4310-bf3f-f02e44f0b669 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:33.249140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101ec6-a41a-4310-bf3f-f02e44f0b669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:33.268312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-53101ec6-a41a-4310-bf3f-f02e44f0b669 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:33.269516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-53101ec6-a41a-4310-bf3f-f02e44f0b669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:33.343824 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 480/962] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:33.623470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74638e3f-4f93-4a1d-9d19-cd621fd489b8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:33.625772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ce5b7-4058-4f2e-8c5a-4ddc0c2875b1 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:33.626042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ce5b7-4058-4f2e-8c5a-4ddc0c2875b1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:33.627012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ce5b7-4058-4f2e-8c5a-4ddc0c2875b1 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:33.629381 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 481/963] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:33.634911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea7231-1231-400f-960a-0a1a45a9a1be None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:33.635195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea7231-1231-400f-960a-0a1a45a9a1be None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:33.635462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea7231-1231-400f-960a-0a1a45a9a1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:33.654268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0dea7231-1231-400f-960a-0a1a45a9a1be None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:33.655540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0dea7231-1231-400f-960a-0a1a45a9a1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:33.720953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:33.734019 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 482/964] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:33.746364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74638e3f-4f93-4a1d-9d19-cd621fd489b8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:33.783493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54667cd5-3b25-47df-b679-9c66246c4e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:33.784132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54667cd5-3b25-47df-b679-9c66246c4e08 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:33.784411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54667cd5-3b25-47df-b679-9c66246c4e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:33.784685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54667cd5-3b25-47df-b679-9c66246c4e08 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:33.795341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54667cd5-3b25-47df-b679-9c66246c4e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-54667cd5-3b25-47df-b679-9c66246c4e08 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:33.800977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:33.801938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:33.803141 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 483/965] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 30 09:23:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:33.808468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea7231-1231-400f-960a-0a1a45a9a1be None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:33.824664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:33.914638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea7231-1231-400f-960a-0a1a45a9a1be None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:33.949176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144e03-6512-4515-8433-262959a68e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:33.949816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144e03-6512-4515-8433-262959a68e7d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:33.950098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144e03-6512-4515-8433-262959a68e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:33.950372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144e03-6512-4515-8433-262959a68e7d None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:33.951032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35144e03-6512-4515-8433-262959a68e7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:33.951335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35144e03-6512-4515-8433-262959a68e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:33.952714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35144e03-6512-4515-8433-262959a68e7d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:33.961760 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 483/966] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 30 09:23:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:33.967356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54667cd5-3b25-47df-b679-9c66246c4e08 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.029937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144e03-6512-4515-8433-262959a68e7d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.106577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54667cd5-3b25-47df-b679-9c66246c4e08 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:34.114974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:34.140596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144e03-6512-4515-8433-262959a68e7d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:34.144948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:34.145585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.145895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.146171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.148979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:34.159738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:34.160210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:34.176456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:34.177420 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:34.178333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:34.178333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:34.178333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:34.181896 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 484/967] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:34.183684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.185309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:34.186011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.186414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.186733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.187571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:34.189568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:34.192360 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING keystone.server.flask.common [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 30 09:23:34.206202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.sql.core [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 30 09:23:34.206202 dv-cinder-master-3-smb devstack@keystone.service[81371]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 30 09:23:34.206202 dv-cinder-master-3-smb devstack@keystone.service[81371]: [parameters: {'user_id': '6bb744ef7c694c75a9ee55846e0f3c8d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 30 09:23:34.206202 dv-cinder-master-3-smb devstack@keystone.service[81371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Sep 30 09:23:34.214609 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 09:23:34.215475 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application yield resource Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.rollback() Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.force_reraise() Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise self.value Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application session.commit() Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 30 09:23:34.217173 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._prepare_impl() Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.flush() Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._flush(objects) Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application compat.raise_( Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application flush_context.execute() Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rec.execute(self) Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 30 09:23:34.218306 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 30 09:23:34.219442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 30 09:23:34.221197 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [parameters: {'user_id': '6bb744ef7c694c75a9ee55846e0f3c8d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:34.222219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 30 09:23:34.223177 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 30 09:23:34.224263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 30 09:23:34.224263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 30 09:23:34.224263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 30 09:23:34.224263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 30 09:23:34.224263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 30 09:23:34.224263 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:34.224263 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 484/968] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 30 09:23:34 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:34.237927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.300623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ad495-a236-43e9-a603-3e84afcac696 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:34.342830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:34.343497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.343872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.344174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.344808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:34.345141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:34.346810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:34.365785 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 485/969] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:34.367376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.387316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd5b4d-e525-416c-b487-a71bf8d49bf7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:34.428549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:34.429967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4319d0-81d9-43e7-a67b-70684dfcb6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:34.430617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4319d0-81d9-43e7-a67b-70684dfcb6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.430908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4319d0-81d9-43e7-a67b-70684dfcb6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.431179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4319d0-81d9-43e7-a67b-70684dfcb6f5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.441415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4319d0-81d9-43e7-a67b-70684dfcb6f5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:34.443552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4319d0-81d9-43e7-a67b-70684dfcb6f5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:34.445524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4d4319d0-81d9-43e7-a67b-70684dfcb6f5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:34.457786 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:34.458920 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 30 09:23:34.459876 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:34.463470 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 485/970] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:34.464667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebaf6b-b82f-4c9e-8208-528e2e3f2604 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.465322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebaf6b-b82f-4c9e-8208-528e2e3f2604 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.465958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebaf6b-b82f-4c9e-8208-528e2e3f2604 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.470902 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 486/971] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:34.471340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebaf6b-b82f-4c9e-8208-528e2e3f2604 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.494547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7fe95-c179-458b-abd9-04befc7a9669 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:34.530255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fbb13-6454-4b34-9841-1759a8e457ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:34.530880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fbb13-6454-4b34-9841-1759a8e457ba None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.531154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fbb13-6454-4b34-9841-1759a8e457ba None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.531426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fbb13-6454-4b34-9841-1759a8e457ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.539189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fbb13-6454-4b34-9841-1759a8e457ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:34.540628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fbb13-6454-4b34-9841-1759a8e457ba None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-4f4fbb13-6454-4b34-9841-1759a8e457ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:34.552930 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:34.553933 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:34.554906 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 486/972] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:23:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:34.556007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8bd69-f95c-40f9-a2ba-27bdeb672c81 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.556007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8bd69-f95c-40f9-a2ba-27bdeb672c81 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.556174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8bd69-f95c-40f9-a2ba-27bdeb672c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.573947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d1d8bd69-f95c-40f9-a2ba-27bdeb672c81 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:34.575124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d1d8bd69-f95c-40f9-a2ba-27bdeb672c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:34.583328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebaf6b-b82f-4c9e-8208-528e2e3f2604 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:34.617408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fcce2-aff0-4d42-87ec-7aaee18fdb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:34.618048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fcce2-aff0-4d42-87ec-7aaee18fdb9e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.618335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fcce2-aff0-4d42-87ec-7aaee18fdb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.618611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fcce2-aff0-4d42-87ec-7aaee18fdb9e None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.619315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fcce2-aff0-4d42-87ec-7aaee18fdb9e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:34.619618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fcce2-aff0-4d42-87ec-7aaee18fdb9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:34.621074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fcce2-aff0-4d42-87ec-7aaee18fdb9e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:34.653233 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 487/973] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:34.654438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8bd69-f95c-40f9-a2ba-27bdeb672c81 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.661925 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 487/974] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 30 09:23:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:34.707653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:34.776660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8bd69-f95c-40f9-a2ba-27bdeb672c81 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:34.815628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:34.816597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.816727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733f2de-bae2-4f40-8b04-f778fad59693 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.816978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733f2de-bae2-4f40-8b04-f778fad59693 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.817256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733f2de-bae2-4f40-8b04-f778fad59693 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.817621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.818136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.818935 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 488/975] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:23:34.819334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:34.819829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:34.821537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:34.824504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c1ec-7cc2-4def-84ff-0551b0b2d143 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:34.824790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c1ec-7cc2-4def-84ff-0551b0b2d143 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:34.825071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c1ec-7cc2-4def-84ff-0551b0b2d143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:34.844488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3665c1ec-7cc2-4def-84ff-0551b0b2d143 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:34.845742 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 488/976] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 30 09:23:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:34.850776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3665c1ec-7cc2-4def-84ff-0551b0b2d143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:34.853967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.917016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:34.937422 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 489/977] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:34.938360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665c1ec-7cc2-4def-84ff-0551b0b2d143 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:34.983871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:34.989487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47ccf-be5c-48c0-bbec-d48fb46e5fb0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.028261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ebab4-3978-4443-aed0-85f4cf251291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.028874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ebab4-3978-4443-aed0-85f4cf251291 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.029293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ebab4-3978-4443-aed0-85f4cf251291 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.029404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ebab4-3978-4443-aed0-85f4cf251291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.052775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665c1ec-7cc2-4def-84ff-0551b0b2d143 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.088581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1fd06-f16d-4b68-899d-efd4b7360808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.089179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1fd06-f16d-4b68-899d-efd4b7360808 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.089452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1fd06-f16d-4b68-899d-efd4b7360808 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.089755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1fd06-f16d-4b68-899d-efd4b7360808 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.090514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1fd06-f16d-4b68-899d-efd4b7360808 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.092002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1fd06-f16d-4b68-899d-efd4b7360808 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.108825 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 490/978] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:23:34 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:35.109001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ebab4-3978-4443-aed0-85f4cf251291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.110420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ebab4-3978-4443-aed0-85f4cf251291 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.217145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:35.223024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1fd06-f16d-4b68-899d-efd4b7360808 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.227480 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 489/979] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:34 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:23:35.232877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ebab4-3978-4443-aed0-85f4cf251291 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.275157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:35.346271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1fd06-f16d-4b68-899d-efd4b7360808 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.348775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ebab4-3978-4443-aed0-85f4cf251291 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.383198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.384301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.384588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.384859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.386065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.386689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.386860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.386954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.387226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.388252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.404975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.404975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:35.405494 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:35.406488 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:35.407324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:35.409673 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 491/980] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:35.415688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:35.434901 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:35.436092 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:35.437085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:35.437085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:35.437085 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:35.437374 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 490/981] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:35.443418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.499329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:35.533332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650253a6-039d-4081-88cd-33033bbf3b41 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.541442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:35.562522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0fbfd-4cba-480e-ab06-9bc6018e0fd8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.572522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.573448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.573951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.574406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.575349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:35.575850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.577448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.584274 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 492/982] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:35.589652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.603367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.604250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.604529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.604798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.605607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:35.605934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.608170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.625675 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 491/983] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:35.628777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.698381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71882e24-9e27-47d3-90a7-48875fb83565 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.736266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.737164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.737409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.737699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.738322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:35.738612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.739987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.741345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc87ea-f3dd-4326-8cb5-0a5d0d332288 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.749732 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 493/984] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:35.751653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.752819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:35.788519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:35.788912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.789877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.789877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.790137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.797317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.799356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:35.810885 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:35.811790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:35.812696 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 492/985] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:35.818194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.862594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725ecc-9509-4f3a-959d-166f51b4f752 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.899076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.899681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.899962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.900233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.901210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] RBAC: Adding query filter params (interface=admin, service_id=dcf7d8cdf2e941b2875648c9d41b1898, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:35.901522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.902940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.913753 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 494/986] 10.222.0.15 () {56 vars in 1249 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/endpoints?service_id=dcf7d8cdf2e941b2875648c9d41b1898&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:35.914760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:35.934728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880e530-35d4-49ec-a763-24ccc045551f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:35.971274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:35.971881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:35.972159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:35.972431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:35.973110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:35.973412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:35.974865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:35.987273 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 493/987] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:35.993089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:36.019216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:36.023446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:36.024282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7dc5-4130-4c11-83f3-a0eed1b764d7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:36.069254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b44c6d-cd61-473b-ae4d-22a0e3437b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:36.071887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44c6d-cd61-473b-ae4d-22a0e3437b0f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:36.071887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44c6d-cd61-473b-ae4d-22a0e3437b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:36.071887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44c6d-cd61-473b-ae4d-22a0e3437b0f None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:36.071887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b44c6d-cd61-473b-ae4d-22a0e3437b0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:36.072348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b44c6d-cd61-473b-ae4d-22a0e3437b0f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:36.080721 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 495/988] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:23:36.123873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1599b67d-0fc2-4024-8177-ea63f093b065 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:36.164839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bbead-f154-4727-a57f-097888585e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:36.165814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bbead-f154-4727-a57f-097888585e66 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:36.166248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bbead-f154-4727-a57f-097888585e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:36.166670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bbead-f154-4727-a57f-097888585e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:36.167591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bbead-f154-4727-a57f-097888585e66 None admin] RBAC: Adding query filter params (role.id=61285fdf54ab4933b5e0b2f767155362, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=29179e1dc1b045a08cbd7feadc2ececd) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:36.168052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bbead-f154-4727-a57f-097888585e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:36.169699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bbead-f154-4727-a57f-097888585e66 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:36.186141 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 494/989] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:35 2022] GET /identity/v3/role_assignments?user.id=29179e1dc1b045a08cbd7feadc2ececd&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=61285fdf54ab4933b5e0b2f767155362 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:37.404009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4ced5-4868-4fc4-bd16-158d58e5be0d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:37.404526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4ced5-4868-4fc4-bd16-158d58e5be0d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:37.404635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4ced5-4868-4fc4-bd16-158d58e5be0d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:37.406218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 496/990] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:37.412504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f3f43-16f9-4ec9-8823-31f12faeef6f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:37.412923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f3f43-16f9-4ec9-8823-31f12faeef6f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:37.413028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f3f43-16f9-4ec9-8823-31f12faeef6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:37.430913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-309f3f43-16f9-4ec9-8823-31f12faeef6f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:37.432081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-309f3f43-16f9-4ec9-8823-31f12faeef6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:37.502687 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 495/991] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:37.775181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4ced5-4868-4fc4-bd16-158d58e5be0d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:37.881876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4ced5-4868-4fc4-bd16-158d58e5be0d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:37.918428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:37.919082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:37.919364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:37.919635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:37.920210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:37.921683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:37.931057 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:37.932049 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:37.934684 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 497/992] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:37.938094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f3f43-16f9-4ec9-8823-31f12faeef6f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:37.963864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:38.041548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f3f43-16f9-4ec9-8823-31f12faeef6f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:38.076081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:38.076689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.076968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.077239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.077898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] RBAC: Adding query filter params (name=compute) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:38.078201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:38.080109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:38.089821 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 496/993] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:38.090727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:38.119307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:38.193881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe8c33-eef5-4b68-afbf-3ba19d7cc49a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:38.231672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1057811-eaab-4575-9d45-0e0c334fa8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:38.232295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057811-eaab-4575-9d45-0e0c334fa8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.232573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057811-eaab-4575-9d45-0e0c334fa8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.232842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057811-eaab-4575-9d45-0e0c334fa8f5 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.233488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1057811-eaab-4575-9d45-0e0c334fa8f5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:38.233821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1057811-eaab-4575-9d45-0e0c334fa8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:38.235239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1057811-eaab-4575-9d45-0e0c334fa8f5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:38.242079 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 498/994] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:38.247887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:38.337870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:38.343049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889ddd0-1a06-4752-8b5c-1042ccb036a4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.343393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889ddd0-1a06-4752-8b5c-1042ccb036a4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.343611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889ddd0-1a06-4752-8b5c-1042ccb036a4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.345686 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 499/995] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:38.351513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b851c-ab0d-430d-bcb6-e2dde74ca6ca None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.351799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b851c-ab0d-430d-bcb6-e2dde74ca6ca None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.352073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b851c-ab0d-430d-bcb6-e2dde74ca6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.357829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9023c-68fd-423f-a80c-593b1dbe2abf None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:38.370436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a67b851c-ab0d-430d-bcb6-e2dde74ca6ca None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:38.371629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a67b851c-ab0d-430d-bcb6-e2dde74ca6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:38.397374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:38.398040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.398396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.398675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.399572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] RBAC: Adding query filter params (interface=public, service_id=e296a0b2a3f84cb28dfb671d4842e451, region_id=RegionOne) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:38.399888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:38.401914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:38.408389 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 497/996] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 30 09:23:38 2022] GET /identity/v3/endpoints?service_id=e296a0b2a3f84cb28dfb671d4842e451&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:38.413478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:38.449419 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 500/997] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:38.450715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a341-6502-49c8-8614-ba3f889c51da None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.450993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a341-6502-49c8-8614-ba3f889c51da None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.451257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a341-6502-49c8-8614-ba3f889c51da None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.452633 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 501/998] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:38.458266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa30167-d8b1-4255-b919-1c74b85aeb62 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.458546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa30167-d8b1-4255-b919-1c74b85aeb62 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.458833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa30167-d8b1-4255-b919-1c74b85aeb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.478579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3aa30167-d8b1-4255-b919-1c74b85aeb62 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:38.479770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3aa30167-d8b1-4255-b919-1c74b85aeb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:38.495358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:38.520001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cd138-c831-4a3f-96b2-a3e60aabdc65 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:38.555028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8afeed-8e31-4f0a-9ac4-49da55d210b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:38.555758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8afeed-8e31-4f0a-9ac4-49da55d210b5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.556037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8afeed-8e31-4f0a-9ac4-49da55d210b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.556308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8afeed-8e31-4f0a-9ac4-49da55d210b5 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.556909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8afeed-8e31-4f0a-9ac4-49da55d210b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:38.558326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8afeed-8e31-4f0a-9ac4-49da55d210b5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:38.559121 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 502/999] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:38.573723 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 498/1000] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:23:38 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:23:38.725933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa30167-d8b1-4255-b919-1c74b85aeb62 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:38.825547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8afeed-8e31-4f0a-9ac4-49da55d210b5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:38.829752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa30167-d8b1-4255-b919-1c74b85aeb62 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:38.843302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:38.866821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:38.867460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.867741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.868015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.868599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:38.870086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:38.880122 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:38.881112 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:38.881959 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 503/1001] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 30 09:23:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:38.886981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:38.904529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:38.937119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8afeed-8e31-4f0a-9ac4-49da55d210b5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:38.972586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:38.973218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:38.973496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:38.973813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] PATH_INFO: `/v3/services/image` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:38.974391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:38.975762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:38.984758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:38.986316 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:38.987150 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 499/1002] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 30 09:23:38 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:38.992233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:39.013739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09994be9-74bc-46b8-aadb-4699a36b4ca8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:39.049695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:39.050311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:39.050620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:39.050871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:39.051505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:39.051809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:39.053193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:39.058118 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 504/1003] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:39.063686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:39.087109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:39.114234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc95cc7-1264-46fa-8c1d-f51d0bcaef8f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:39.145837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:39.151558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:39.152176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:39.152453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:39.152726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:39.153350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:39.155180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:39.156079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:39.161871 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 500/1004] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:39.167005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:39.174729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e3f1b-9fcd-4d7f-84f4-a174616af703 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:39.212118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:39.212726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:39.213004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:39.213273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:39.213929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:39.214233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:39.215611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:39.221231 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 505/1005] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 30 09:23:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:39.226564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:39.291723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb02d-7fcf-4759-b831-6ee31a64d4a2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:39.327121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c3657-7310-414b-a8d5-b5fc9afd2752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:39.327735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c3657-7310-414b-a8d5-b5fc9afd2752 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:39.328015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c3657-7310-414b-a8d5-b5fc9afd2752 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:39.328285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c3657-7310-414b-a8d5-b5fc9afd2752 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:39.328912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c3657-7310-414b-a8d5-b5fc9afd2752 None admin] RBAC: Adding query filter params (type=image) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:39.329292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c3657-7310-414b-a8d5-b5fc9afd2752 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:39.330763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c3657-7310-414b-a8d5-b5fc9afd2752 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:39.331037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a679afb-ad73-4e8a-8506-7563d718c651 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:39.335828 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 501/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:39.375014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246cc703-0fba-48d8-8145-c75f8973d97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:39.375646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc703-0fba-48d8-8145-c75f8973d97b None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:39.375929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc703-0fba-48d8-8145-c75f8973d97b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:39.376792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246cc703-0fba-48d8-8145-c75f8973d97b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:39.377481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cc703-0fba-48d8-8145-c75f8973d97b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:39.379013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246cc703-0fba-48d8-8145-c75f8973d97b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:39.402940 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 506/1007] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:23:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:23:40.735208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c79c-1e59-4436-8b26-69c1a8c6a31c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:40.736208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c79c-1e59-4436-8b26-69c1a8c6a31c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:40.736313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c79c-1e59-4436-8b26-69c1a8c6a31c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:40.737827 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 502/1008] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:40.742787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27409782-96ff-4afe-9be1-7582d3869cb2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:40.743170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27409782-96ff-4afe-9be1-7582d3869cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:40.743339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27409782-96ff-4afe-9be1-7582d3869cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:40.749836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:40.763402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-27409782-96ff-4afe-9be1-7582d3869cb2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:40.764570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-27409782-96ff-4afe-9be1-7582d3869cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:40.837770 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 507/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:41.104459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6c79c-1e59-4436-8b26-69c1a8c6a31c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:41.114530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:41.213017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6c79c-1e59-4436-8b26-69c1a8c6a31c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:41.246310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:41.246939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:41.247214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:41.247482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:41.248051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:41.249402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:41.258553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:41.259520 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:41.260396 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 503/1010] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:41.269218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27409782-96ff-4afe-9be1-7582d3869cb2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:41.368634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27409782-96ff-4afe-9be1-7582d3869cb2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:41.399068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:41.413068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42d313-9e63-4261-90a1-b0cac1094b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:41.413741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d313-9e63-4261-90a1-b0cac1094b7f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:41.414021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d313-9e63-4261-90a1-b0cac1094b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:41.414290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d313-9e63-4261-90a1-b0cac1094b7f None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:41.414917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42d313-9e63-4261-90a1-b0cac1094b7f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:41.415218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42d313-9e63-4261-90a1-b0cac1094b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:41.416590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42d313-9e63-4261-90a1-b0cac1094b7f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:41.426308 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 508/1011] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:41.427287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:41.503777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:41.538793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92948731-ccda-43c4-8341-9c802160ac55 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:41.571941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:41.572914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:41.572914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:41.573178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:41.573819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:41.574119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:41.574407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c9f2a-8068-46d4-af47-0f9451af9752 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:41.574697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c9f2a-8068-46d4-af47-0f9451af9752 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:41.574972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c9f2a-8068-46d4-af47-0f9451af9752 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:41.575469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:41.576455 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 509/1012] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:41.581846 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 504/1013] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 30 09:23:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:41.582358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6b3a5-c130-43d8-ba79-42b5aa575df5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:41.582644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6b3a5-c130-43d8-ba79-42b5aa575df5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:41.582923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6b3a5-c130-43d8-ba79-42b5aa575df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:41.591558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:41.602445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9cb6b3a5-c130-43d8-ba79-42b5aa575df5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:41.603638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9cb6b3a5-c130-43d8-ba79-42b5aa575df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:41.671014 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 510/1014] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:41.672275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf515f5-8e41-4570-ab2d-98cab2d0c9e6 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:41.672563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf515f5-8e41-4570-ab2d-98cab2d0c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:41.672862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf515f5-8e41-4570-ab2d-98cab2d0c9e6 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:41.674180 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 511/1015] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:41.679677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e6e47-0ab6-4082-bc3d-264aa35d59fb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:41.679960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e6e47-0ab6-4082-bc3d-264aa35d59fb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:41.680233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e6e47-0ab6-4082-bc3d-264aa35d59fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:41.694495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca769c5-f5ca-4eb7-8bb1-1c1491753313 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:41.699037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a59e6e47-0ab6-4082-bc3d-264aa35d59fb None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:41.700237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a59e6e47-0ab6-4082-bc3d-264aa35d59fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:41.729560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00055b76-ea1c-4751-9427-8edae4e70093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:41.730191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00055b76-ea1c-4751-9427-8edae4e70093 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:41.730467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00055b76-ea1c-4751-9427-8edae4e70093 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:41.730738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00055b76-ea1c-4751-9427-8edae4e70093 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:41.731396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00055b76-ea1c-4751-9427-8edae4e70093 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:41.732744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00055b76-ea1c-4751-9427-8edae4e70093 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:41.737484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:41.761661 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 505/1016] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:23:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:23:41.785898 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 512/1017] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:41.945159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00055b76-ea1c-4751-9427-8edae4e70093 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:42.052255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00055b76-ea1c-4751-9427-8edae4e70093 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:42.059977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e6e47-0ab6-4082-bc3d-264aa35d59fb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:42.088320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654bdfe-2f1f-48ea-a728-af41492a4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:42.089267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654bdfe-2f1f-48ea-a728-af41492a4b7d None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:42.089557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654bdfe-2f1f-48ea-a728-af41492a4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:42.090060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654bdfe-2f1f-48ea-a728-af41492a4b7d None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:42.090657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654bdfe-2f1f-48ea-a728-af41492a4b7d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:42.092537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654bdfe-2f1f-48ea-a728-af41492a4b7d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:42.112834 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 506/1018] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:23:41 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:42.117673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:42.177571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e6e47-0ab6-4082-bc3d-264aa35d59fb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:42.222576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dc3b9-fff1-4995-a4f6-da82a27b3de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:42.223137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc3b9-fff1-4995-a4f6-da82a27b3de4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:42.223419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc3b9-fff1-4995-a4f6-da82a27b3de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:42.223690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc3b9-fff1-4995-a4f6-da82a27b3de4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:42.224828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dc3b9-fff1-4995-a4f6-da82a27b3de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:42.226355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dc3b9-fff1-4995-a4f6-da82a27b3de4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:42.228178 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 513/1019] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:42.233891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654bdfe-2f1f-48ea-a728-af41492a4b7d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:42.289010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:42.348903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654bdfe-2f1f-48ea-a728-af41492a4b7d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:42.384355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d4be6-f176-466e-a6ca-4f9910c7470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:42.385013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4be6-f176-466e-a6ca-4f9910c7470d None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:42.385301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4be6-f176-466e-a6ca-4f9910c7470d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:42.385653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4be6-f176-466e-a6ca-4f9910c7470d None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:42.386321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d4be6-f176-466e-a6ca-4f9910c7470d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:42.387755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d4be6-f176-466e-a6ca-4f9910c7470d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:42.424040 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 507/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:23:42 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:44.038331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e82538-3433-4cc7-81c6-cd0a927fbd6a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.038856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e82538-3433-4cc7-81c6-cd0a927fbd6a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.038856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e82538-3433-4cc7-81c6-cd0a927fbd6a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.040195 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 514/1021] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:44.045563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b692d1-8d54-4e99-8dbe-a9a624c06128 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.046023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b692d1-8d54-4e99-8dbe-a9a624c06128 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.046134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b692d1-8d54-4e99-8dbe-a9a624c06128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.064836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-35b692d1-8d54-4e99-8dbe-a9a624c06128 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:44.066042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-35b692d1-8d54-4e99-8dbe-a9a624c06128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:44.132651 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 508/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:44.326495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd018-7667-429f-bcb8-35e62bcd1d45 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.326680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd018-7667-429f-bcb8-35e62bcd1d45 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.326961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd018-7667-429f-bcb8-35e62bcd1d45 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.328314 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 515/1023] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:44.334190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22344629-9b8c-4116-a878-ea2935c909b6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.334479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22344629-9b8c-4116-a878-ea2935c909b6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.334752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22344629-9b8c-4116-a878-ea2935c909b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.353258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-22344629-9b8c-4116-a878-ea2935c909b6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:44.354452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-22344629-9b8c-4116-a878-ea2935c909b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:44.380380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:44.409411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfd018-7667-429f-bcb8-35e62bcd1d45 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:44.429554 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 509/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:44.519931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfd018-7667-429f-bcb8-35e62bcd1d45 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:44.534912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:44.560271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b6480-6a25-49e0-bd1a-66b71036abb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:44.560919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b6480-6a25-49e0-bd1a-66b71036abb2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.561201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b6480-6a25-49e0-bd1a-66b71036abb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.561473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b6480-6a25-49e0-bd1a-66b71036abb2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.572882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b6480-6a25-49e0-bd1a-66b71036abb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5e0b6480-6a25-49e0-bd1a-66b71036abb2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:44.577698 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:44.578633 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:44.580403 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 516/1025] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 30 09:23:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:44.584782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22344629-9b8c-4116-a878-ea2935c909b6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:44.651270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f74f71-e208-4785-851a-24692a6b0877 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.651568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f74f71-e208-4785-851a-24692a6b0877 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.651818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f74f71-e208-4785-851a-24692a6b0877 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.653879 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 517/1026] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:23:44.659074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca9a3-1977-4954-9072-59a3de2c8f55 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.659537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca9a3-1977-4954-9072-59a3de2c8f55 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.659974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ca9a3-1977-4954-9072-59a3de2c8f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.677914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-859ca9a3-1977-4954-9072-59a3de2c8f55 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:44.679108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-859ca9a3-1977-4954-9072-59a3de2c8f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:44.687603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22344629-9b8c-4116-a878-ea2935c909b6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:44.722265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:44.722875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.723151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.723419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.724077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:44.724384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:44.725759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:44.733877 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 510/1027] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 30 09:23:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:44.734529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:44.749534 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 518/1028] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:44.750409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca9a3-1977-4954-9072-59a3de2c8f55 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:44.762181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:44.845347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb95e8-b0cc-473a-974b-de1cf0e0f01f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:44.861814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ca9a3-1977-4954-9072-59a3de2c8f55 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:44.863527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:44.884588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:44.885220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.885497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.885882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.886376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:44.887740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:44.897362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:44.898553 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:44.899564 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:44.900742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:44.900742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:44.900742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:44.900742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:44.902530 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 511/1029] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 30 09:23:44 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:44.908594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:44.914704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:44.916156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:44.931327 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:44.932363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:44.933258 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:44.933258 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:44.933258 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:44.933258 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 519/1030] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:44.938212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.004137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:45.020831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ad863-a192-4cea-8c71-2f476787acc9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.046381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57856e-1cfd-495e-a97c-32bb5463533a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.057760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.058364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.058638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.058904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.059522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:45.059822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.061171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.066948 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 512/1031] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:45.067905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.084108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.084715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.084992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.085261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.085931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:45.086234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.087609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.099614 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 520/1032] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:45.100831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.120590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:45.173605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b216e62-9b98-4b3f-a5d2-032f75bc15a5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.206902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19eab-f6b3-4ade-86be-a6a145fc352c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.209046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75564e-4f0f-48c5-97a2-6a1db50b007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.209691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75564e-4f0f-48c5-97a2-6a1db50b007e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.209970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75564e-4f0f-48c5-97a2-6a1db50b007e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.210237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75564e-4f0f-48c5-97a2-6a1db50b007e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.211326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75564e-4f0f-48c5-97a2-6a1db50b007e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.212774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75564e-4f0f-48c5-97a2-6a1db50b007e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.214471 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 513/1033] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:45.215227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75564e-4f0f-48c5-97a2-6a1db50b007e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.237368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:45.245467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.246107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.246387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.246666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.247296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:45.247600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.248982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.254109 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 521/1034] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:45.254883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.315174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75564e-4f0f-48c5-97a2-6a1db50b007e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.334636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:45.349583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.350288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.350511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.350779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.356485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6a79d-7bd8-462b-a588-fbad77b761a7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.357553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.359008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:45.371093 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:45.372210 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 514/1035] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:23:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:45.373208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.393032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6128-7dcf-46f1-802b-b3f06bf94c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.393651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6128-7dcf-46f1-802b-b3f06bf94c60 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.393942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6128-7dcf-46f1-802b-b3f06bf94c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.394211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6128-7dcf-46f1-802b-b3f06bf94c60 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.394856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c6128-7dcf-46f1-802b-b3f06bf94c60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.396292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c6128-7dcf-46f1-802b-b3f06bf94c60 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.420732 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 522/1036] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:23:45 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:23:45.421850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6128-7dcf-46f1-802b-b3f06bf94c60 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.473917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:45.497007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6149a-729b-4a66-9ab9-6fa10a08f33e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.540711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.541428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.541768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.541966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.542680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] RBAC: Adding query filter params (interface=public, service_id=988f01e5440646f8bdc2d4c8fa78385d, region_id=RegionOne) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:45.543170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.544935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.550856 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 515/1037] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 30 09:23:45 2022] GET /identity/v3/endpoints?service_id=988f01e5440646f8bdc2d4c8fa78385d&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:45.556796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.558002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6128-7dcf-46f1-802b-b3f06bf94c60 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.598355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.598977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.599253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.599531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.600263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:45.600833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.602154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.603809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:45.623711 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 523/1038] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:23:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:45.631088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:45.679738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc726c88-3e12-47f1-855d-15247e473b3d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.717101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3caaf-0e8b-43e3-bbb7-ea8fec1f06ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.718128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3caaf-0e8b-43e3-bbb7-ea8fec1f06ea None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.718406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3caaf-0e8b-43e3-bbb7-ea8fec1f06ea None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.718678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3caaf-0e8b-43e3-bbb7-ea8fec1f06ea None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.719799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3caaf-0e8b-43e3-bbb7-ea8fec1f06ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.721607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3caaf-0e8b-43e3-bbb7-ea8fec1f06ea None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.728684 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 516/1039] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:23:45 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:23:45.745869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3801bd4-ed68-4768-8a47-bbb5ac82344f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:45.792491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e43d9-0b91-4419-af08-c2d09c560af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:45.793438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e43d9-0b91-4419-af08-c2d09c560af3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:45.793957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e43d9-0b91-4419-af08-c2d09c560af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:45.794440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e43d9-0b91-4419-af08-c2d09c560af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:45.860225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:45.872236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e43d9-0b91-4419-af08-c2d09c560af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:45.873768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e43d9-0b91-4419-af08-c2d09c560af3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:45.985700 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 524/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 09:23:45.988650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3caaf-0e8b-43e3-bbb7-ea8fec1f06ea None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:46.001554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:46.100124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3caaf-0e8b-43e3-bbb7-ea8fec1f06ea None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:46.134081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:46.134719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:46.134999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:46.135272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:46.146124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:46.147573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:46.162388 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:46.163353 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:46.164282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:46.164282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:46.164282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:46.165709 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 517/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:46.169204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e43d9-0b91-4419-af08-c2d09c560af3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:46.262410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:46.272206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e43d9-0b91-4419-af08-c2d09c560af3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:46.306802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:46.307408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:46.307683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:46.307951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:46.308582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:46.308883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:46.310281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:46.325676 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 525/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:46.327882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:46.398424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:46.441686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73b077-cfaa-406b-a62b-dbf0a0ab848b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:46.475639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:46.476267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:46.476542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:46.476809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:46.483736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:46.485744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:46.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:46.498466 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:46.499351 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:46.499351 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:46.499351 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:46.499351 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:46.499351 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:46.499351 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:46.499351 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:46.499351 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:46.502039 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 518/1043] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:46.505001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:46.608388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caa78f-8ef7-49ee-a6ca-5e68c3fb3f6d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:46.647029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c13fa2-a717-4c5f-84f5-d8981796d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:46.647645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c13fa2-a717-4c5f-84f5-d8981796d7cb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:46.647928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c13fa2-a717-4c5f-84f5-d8981796d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:46.648202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c13fa2-a717-4c5f-84f5-d8981796d7cb None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:46.648950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c13fa2-a717-4c5f-84f5-d8981796d7cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:46.649203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c13fa2-a717-4c5f-84f5-d8981796d7cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:46.650915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c13fa2-a717-4c5f-84f5-d8981796d7cb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:46.663871 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 526/1044] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:46.669722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:46.776151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666f9f9-df1e-4a18-8d87-1c5e842636a2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:46.795680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:46.811885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c54be9-9fa7-480a-a222-3e6bf9b0376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:46.812487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c54be9-9fa7-480a-a222-3e6bf9b0376e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:46.812765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c54be9-9fa7-480a-a222-3e6bf9b0376e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:46.813035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c54be9-9fa7-480a-a222-3e6bf9b0376e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:46.814936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c54be9-9fa7-480a-a222-3e6bf9b0376e None admin] RBAC: Adding query filter params (role.id=61285fdf54ab4933b5e0b2f767155362, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=aa055ee122ce4e92b4da7f8222521a9e) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:46.815244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c54be9-9fa7-480a-a222-3e6bf9b0376e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:46.816657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c54be9-9fa7-480a-a222-3e6bf9b0376e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:46.822116 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 519/1045] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:46 2022] GET /identity/v3/role_assignments?user.id=aa055ee122ce4e92b4da7f8222521a9e&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=61285fdf54ab4933b5e0b2f767155362 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:47.685541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11751f7c-d1ce-4760-a6ca-7f12b8b35b3b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:47.686062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11751f7c-d1ce-4760-a6ca-7f12b8b35b3b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:47.686062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11751f7c-d1ce-4760-a6ca-7f12b8b35b3b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:47.687534 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 527/1046] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:47.693259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3e062-318c-40ce-9e87-3647172d63ad None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:47.693811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3e062-318c-40ce-9e87-3647172d63ad None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:47.693929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3e062-318c-40ce-9e87-3647172d63ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:47.714246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cfe3e062-318c-40ce-9e87-3647172d63ad None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:47.715479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cfe3e062-318c-40ce-9e87-3647172d63ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:47.791721 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 520/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:47.909748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae65b5-4134-4a89-bc5f-4e008f9fc431 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:47.910103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae65b5-4134-4a89-bc5f-4e008f9fc431 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:47.910284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae65b5-4134-4a89-bc5f-4e008f9fc431 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:47.911761 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 528/1048] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:47.916751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea962f60-2de6-4f92-a770-aeba213dc966 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:47.917041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea962f60-2de6-4f92-a770-aeba213dc966 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:47.917314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea962f60-2de6-4f92-a770-aeba213dc966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:47.935814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ea962f60-2de6-4f92-a770-aeba213dc966 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:47.936907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ea962f60-2de6-4f92-a770-aeba213dc966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:48.003171 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 521/1049] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:48.063001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae65b5-4134-4a89-bc5f-4e008f9fc431 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.078512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:48.167628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae65b5-4134-4a89-bc5f-4e008f9fc431 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:48.202145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95c3f-c0f6-43a3-86e7-f3a7f3170e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:48.202701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95c3f-c0f6-43a3-86e7-f3a7f3170e1f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:48.202984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95c3f-c0f6-43a3-86e7-f3a7f3170e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:48.203256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab95c3f-c0f6-43a3-86e7-f3a7f3170e1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:48.213327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab95c3f-c0f6-43a3-86e7-f3a7f3170e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fab95c3f-c0f6-43a3-86e7-f3a7f3170e1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:48.218137 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:48.219149 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:48.220402 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:48.221952 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 529/1050] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:48.225230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea962f60-2de6-4f92-a770-aeba213dc966 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.268542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95c3f-c0f6-43a3-86e7-f3a7f3170e1f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.270831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:48.328865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea962f60-2de6-4f92-a770-aeba213dc966 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:48.341124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:48.364872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:48.365481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:48.365794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:48.366076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:48.366730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:48.367035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:48.368487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:48.372423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab95c3f-c0f6-43a3-86e7-f3a7f3170e1f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:48.378208 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 522/1051] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:48.383880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.414203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:48.414983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:48.415327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:48.415752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] PATH_INFO: `/v3/services/image` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:48.416667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:48.418523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:48.429823 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:48.430845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:48.434736 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 530/1052] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 30 09:23:48 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:48.437306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.494732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896eb14-8b4b-4f90-b5e7-617cc281df56 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:48.502221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:48.534013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30dc5c-f747-4873-854c-9a436d0e56c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:48.534651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30dc5c-f747-4873-854c-9a436d0e56c3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:48.534922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30dc5c-f747-4873-854c-9a436d0e56c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:48.535188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30dc5c-f747-4873-854c-9a436d0e56c3 None admin] PATH_INFO: `/v3/users/67dd81188d6142dca443cb6b855c1aec` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:48.549216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30dc5c-f747-4873-854c-9a436d0e56c3 None admin] RBAC: Authorizing `identity:get_user(user_id=67dd81188d6142dca443cb6b855c1aec)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:48.550445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30dc5c-f747-4873-854c-9a436d0e56c3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:48.552289 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 523/1053] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:23:48 2022] GET /identity/v3/users/67dd81188d6142dca443cb6b855c1aec => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:48.557820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30dc5c-f747-4873-854c-9a436d0e56c3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.559522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932d551-27e5-40d9-9414-d45ca83af059 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:48.596709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:48.597317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:48.597594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:48.597895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:48.598572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:48.602925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:48.604345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:48.606761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:48.611539 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 531/1054] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:48.616677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.670011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30dc5c-f747-4873-854c-9a436d0e56c3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:48.706823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be15bb-e0c4-4704-a564-f8a1aedca8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:48.707473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be15bb-e0c4-4704-a564-f8a1aedca8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:48.707692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be15bb-e0c4-4704-a564-f8a1aedca8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:48.707965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be15bb-e0c4-4704-a564-f8a1aedca8e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:48.723083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b5006-0bbc-4059-a113-d8c7d9e0cbf3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:48.739747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:48.760150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:48.760963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:48.761250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:48.761524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:48.762185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] RBAC: Adding query filter params (type=image) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:48.762494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:48.763871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:48.769408 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 532/1055] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:48.774788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.785312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be15bb-e0c4-4704-a564-f8a1aedca8e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:48.786875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be15bb-e0c4-4704-a564-f8a1aedca8e0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:48.838666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:48.879156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130e385-b32b-4db7-be4a-9c21ad4ec056 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:48.897835 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 524/1056] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 09:23:48.899099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be15bb-e0c4-4704-a564-f8a1aedca8e0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:48.923137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf659d25-9c0a-40ad-ab1a-21fa1129b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:48.923771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659d25-9c0a-40ad-ab1a-21fa1129b301 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:48.924056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659d25-9c0a-40ad-ab1a-21fa1129b301 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:48.924326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf659d25-9c0a-40ad-ab1a-21fa1129b301 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:48.924996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf659d25-9c0a-40ad-ab1a-21fa1129b301 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:48.926435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf659d25-9c0a-40ad-ab1a-21fa1129b301 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:48.946629 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 533/1057] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:23:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:23:48.979277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:49.011688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4a43-1766-4e4c-89bb-ca8bd1ae7ed7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:49.012222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4a43-1766-4e4c-89bb-ca8bd1ae7ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:49.012668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4a43-1766-4e4c-89bb-ca8bd1ae7ed7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:49.014278 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 534/1058] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:23:49.020940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9920c53-ffe9-4b1b-8220-e31090f20cb3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:49.021345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9920c53-ffe9-4b1b-8220-e31090f20cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:49.021763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9920c53-ffe9-4b1b-8220-e31090f20cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:49.025100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be15bb-e0c4-4704-a564-f8a1aedca8e0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:49.042195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f9920c53-ffe9-4b1b-8220-e31090f20cb3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:49.043530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f9920c53-ffe9-4b1b-8220-e31090f20cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:49.072469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa89978-0de9-4fda-83b8-29b018919f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:49.073544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89978-0de9-4fda-83b8-29b018919f2f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:49.073863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89978-0de9-4fda-83b8-29b018919f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:49.074424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89978-0de9-4fda-83b8-29b018919f2f None admin] PATH_INFO: `/v3/projects/48dd7552ee1a49deba21a14cd14dce54` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:49.075623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa89978-0de9-4fda-83b8-29b018919f2f None admin] RBAC: Authorizing `identity:get_project(project_id=48dd7552ee1a49deba21a14cd14dce54)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:49.077078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa89978-0de9-4fda-83b8-29b018919f2f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:49.081812 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 525/1059] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:23:48 2022] GET /identity/v3/projects/48dd7552ee1a49deba21a14cd14dce54 => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:49.085902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa89978-0de9-4fda-83b8-29b018919f2f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:49.131972 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 535/1060] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:23:49.198595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa89978-0de9-4fda-83b8-29b018919f2f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:49.225129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:49.234901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:49.234901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:49.235164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:49.235467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:49.236221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=48dd7552ee1a49deba21a14cd14dce54, user.id=67dd81188d6142dca443cb6b855c1aec) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:49.236532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:49.237992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:49.243552 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 526/1061] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:49 2022] GET /identity/v3/role_assignments?user.id=67dd81188d6142dca443cb6b855c1aec&scope.project.id=48dd7552ee1a49deba21a14cd14dce54&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:49.401852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9920c53-ffe9-4b1b-8220-e31090f20cb3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:49.452911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:49.511750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9920c53-ffe9-4b1b-8220-e31090f20cb3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:49.546942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914a23a-862e-4816-abe1-7903927957b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:49.547631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a23a-862e-4816-abe1-7903927957b0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:49.547909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a23a-862e-4816-abe1-7903927957b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:49.548409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a23a-862e-4816-abe1-7903927957b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:49.558321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914a23a-862e-4816-abe1-7903927957b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f914a23a-862e-4816-abe1-7903927957b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:49.563163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:49.564128 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:49.564964 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:49.566260 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 536/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 30 09:23:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:49.570379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:49.681445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a802657-da7a-43ef-ad7f-fae7af33164d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:49.715662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:49.718154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:49.718154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:49.718154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:49.718154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:49.718154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:49.719136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:49.727903 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 527/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 30 09:23:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:49.733925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914a23a-862e-4816-abe1-7903927957b0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:49.838429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914a23a-862e-4816-abe1-7903927957b0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:49.849599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:49.874607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:49.875307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:49.875628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:49.875992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:49.886726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:49.888322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:49.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:49.906700 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:49.908114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:49.908114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:49.908114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:49.909951 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 537/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:49.913325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:49.972117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:50.018342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbe49f-ce35-457e-81a2-42adf9ad5f34 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:50.053257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:50.053888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:50.054166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:50.054439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:50.055075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:50.055376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:50.056739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:50.068515 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 528/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:50.074152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:50.176756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983584c3-b11e-44d2-b302-da3152f443d4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:50.210054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:50.210677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:50.210961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:50.211238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:50.218428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:50.220230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:50.231264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:50.232413 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:50.233572 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:50.237668 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 538/1066] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:23:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:50.238680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:50.338765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cbb40-6be7-4f2b-9905-1875f38a3215 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:50.352654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:50.373844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:50.374405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:50.374685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:50.374958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:50.376286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:50.376592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:50.378034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:50.396771 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 529/1067] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:23:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:50.404810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:50.430315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:50.508528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05904e0c-5654-4c1f-a00f-757a6860f949 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:50.542572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:50.543205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:50.543485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:50.543756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:50.554748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:50.556203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:50.570350 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:50.571342 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:50.572272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:50.572272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:50.572272 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:50.573150 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 539/1068] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:50.577106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:50.676460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238ab2d-165a-4370-b353-f611259f7677 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:50.711338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:50.711950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:50.712230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:50.712505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:50.713145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:50.713449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:50.714871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:50.731089 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 530/1069] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:50.733060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:50.814178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:50.843141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572acff0-bc52-4b4f-9c95-33c6c57f0cb7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:50.879331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:50.879965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:50.880243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:50.880512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:50.887223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:50.888678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:50.901394 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:50.902349 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:50.903227 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:50.903227 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:50.903227 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:50.903227 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:50.903227 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:50.903227 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:50.903227 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:50.903227 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:50.908067 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 540/1070] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:50.909460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:50.943168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:51.018043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026f660-0a34-4819-91fb-cf5b58dd6699 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:51.052714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1cc14-262a-4515-8b45-6a29f9bf92a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:51.053325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cc14-262a-4515-8b45-6a29f9bf92a8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:51.053602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cc14-262a-4515-8b45-6a29f9bf92a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:51.053908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cc14-262a-4515-8b45-6a29f9bf92a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:51.054595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1cc14-262a-4515-8b45-6a29f9bf92a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:51.054896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1cc14-262a-4515-8b45-6a29f9bf92a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:51.056308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1cc14-262a-4515-8b45-6a29f9bf92a8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:51.068577 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 531/1071] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:51.074430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:51.182576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a42a94-9932-4ad2-968d-4317cb9c64fe None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:51.216799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d03fc-626a-464c-b59e-08f7cc1ea6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:51.217482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d03fc-626a-464c-b59e-08f7cc1ea6b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:51.217795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d03fc-626a-464c-b59e-08f7cc1ea6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:51.218070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d03fc-626a-464c-b59e-08f7cc1ea6b0 None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/aa055ee122ce4e92b4da7f8222521a9e/roles/61285fdf54ab4933b5e0b2f767155362` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:51.220561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d03fc-626a-464c-b59e-08f7cc1ea6b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=aa055ee122ce4e92b4da7f8222521a9e, role_id=61285fdf54ab4933b5e0b2f767155362)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:51.222057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d03fc-626a-464c-b59e-08f7cc1ea6b0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:51.224377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:51.237671 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 541/1072] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:23:51 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/aa055ee122ce4e92b4da7f8222521a9e/roles/61285fdf54ab4933b5e0b2f767155362 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:23:51.239414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579793bf-57e7-4246-a762-ee288edfd3e1 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:51.239854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579793bf-57e7-4246-a762-ee288edfd3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:51.240281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579793bf-57e7-4246-a762-ee288edfd3e1 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:51.241790 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 532/1073] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:51.247682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4772-00d7-4fc3-9349-83748ff74bb4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:51.248021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4772-00d7-4fc3-9349-83748ff74bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:51.248253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d4772-00d7-4fc3-9349-83748ff74bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:51.269278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-de2d4772-00d7-4fc3-9349-83748ff74bb4 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:51.270563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-de2d4772-00d7-4fc3-9349-83748ff74bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:51.348620 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 542/1074] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:51.411438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485adc2-d738-4772-a6a4-44441d255796 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:51.411720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485adc2-d738-4772-a6a4-44441d255796 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:51.411989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485adc2-d738-4772-a6a4-44441d255796 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:51.413455 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 533/1075] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:51.420396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e3a97-375a-410f-92a1-51c9ccc78013 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:51.420396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e3a97-375a-410f-92a1-51c9ccc78013 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:51.420396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e3a97-375a-410f-92a1-51c9ccc78013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:51.440629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f69e3a97-375a-410f-92a1-51c9ccc78013 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:51.441843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f69e3a97-375a-410f-92a1-51c9ccc78013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:51.510419 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 543/1076] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:51.629372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485adc2-d738-4772-a6a4-44441d255796 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:51.726789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:51.740588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485adc2-d738-4772-a6a4-44441d255796 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:51.775633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d15dd-7f8f-4082-9eda-baa0bf6c4f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:51.776241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d15dd-7f8f-4082-9eda-baa0bf6c4f02 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:51.776517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d15dd-7f8f-4082-9eda-baa0bf6c4f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:51.776783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d15dd-7f8f-4082-9eda-baa0bf6c4f02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:51.793279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e3a97-375a-410f-92a1-51c9ccc78013 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:51.851259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:51.854659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d15dd-7f8f-4082-9eda-baa0bf6c4f02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:51.856219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d15dd-7f8f-4082-9eda-baa0bf6c4f02 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:51.910393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e3a97-375a-410f-92a1-51c9ccc78013 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:51.951683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c9d34-eac0-456c-955a-0231b2e4db74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:51.952324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9d34-eac0-456c-955a-0231b2e4db74 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:51.952604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9d34-eac0-456c-955a-0231b2e4db74 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:51.952717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:51.953234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9d34-eac0-456c-955a-0231b2e4db74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:51.964447 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 534/1077] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:23:51.966332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c9d34-eac0-456c-955a-0231b2e4db74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:51.969890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d15dd-7f8f-4082-9eda-baa0bf6c4f02 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-cf4c9d34-eac0-456c-955a-0231b2e4db74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:51.972185 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:51.973130 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:51.974054 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:51.976358 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 544/1078] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:51.981934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c9d34-eac0-456c-955a-0231b2e4db74 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:52.076124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d15dd-7f8f-4082-9eda-baa0bf6c4f02 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:52.096303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c9d34-eac0-456c-955a-0231b2e4db74 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:52.110459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:52.117968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:52.118657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:52.118938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:52.119199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:52.130275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:52.131743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:52.133699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:52.134332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:52.134617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:52.134892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:52.135579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:52.135886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:52.137324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:52.152143 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 545/1079] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:52.152367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:52.153436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:52.154951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:52.154951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:52.154951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:52.154951 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 535/1080] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:52.156441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:52.156546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:52.204204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:52.263482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8322c7-5d73-44e2-b68d-4456925386d6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:52.263820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c6938-0ed8-40fb-985e-cf9c15d888d1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:52.299020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:52.299630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:52.299912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:52.300184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:52.300820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:52.301122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:52.301653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4bebb-a2f4-45ab-bf08-793995aa617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:52.302276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bebb-a2f4-45ab-bf08-793995aa617d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:52.302516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:52.302610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bebb-a2f4-45ab-bf08-793995aa617d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:52.302811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bebb-a2f4-45ab-bf08-793995aa617d None admin] PATH_INFO: `/v3/users/67dd81188d6142dca443cb6b855c1aec` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:52.303930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4bebb-a2f4-45ab-bf08-793995aa617d None admin] RBAC: Authorizing `identity:get_user(user_id=67dd81188d6142dca443cb6b855c1aec)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:52.305349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4bebb-a2f4-45ab-bf08-793995aa617d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:52.312127 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 546/1081] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:23:52 2022] GET /identity/v3/users/67dd81188d6142dca443cb6b855c1aec => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:52.312547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4bebb-a2f4-45ab-bf08-793995aa617d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:52.316851 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 536/1082] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:52.367698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:52.431515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4bebb-a2f4-45ab-bf08-793995aa617d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:52.471328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c958d-f16a-4dc7-acb3-c613aed8581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:52.471891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c958d-f16a-4dc7-acb3-c613aed8581b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:52.472174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c958d-f16a-4dc7-acb3-c613aed8581b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:52.472450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c958d-f16a-4dc7-acb3-c613aed8581b None admin] PATH_INFO: `/v3/projects/48dd7552ee1a49deba21a14cd14dce54` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:52.473698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c958d-f16a-4dc7-acb3-c613aed8581b None admin] RBAC: Authorizing `identity:get_project(project_id=48dd7552ee1a49deba21a14cd14dce54)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:52.475192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c958d-f16a-4dc7-acb3-c613aed8581b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:52.477079 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 547/1083] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:23:52 2022] GET /identity/v3/projects/48dd7552ee1a49deba21a14cd14dce54 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:52.483011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:52.592130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e1207-520c-4cb9-96d3-4568c5b068b5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:52.594706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:52.630206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d4b2f-e22b-4e21-9033-8d0ea26a37b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:52.630873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d4b2f-e22b-4e21-9033-8d0ea26a37b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:52.631152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d4b2f-e22b-4e21-9033-8d0ea26a37b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:52.631423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d4b2f-e22b-4e21-9033-8d0ea26a37b8 None admin] PATH_INFO: `/v3/projects/48dd7552ee1a49deba21a14cd14dce54/users/67dd81188d6142dca443cb6b855c1aec/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:52.633331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d4b2f-e22b-4e21-9033-8d0ea26a37b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=48dd7552ee1a49deba21a14cd14dce54, user_id=67dd81188d6142dca443cb6b855c1aec, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:52.634789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d4b2f-e22b-4e21-9033-8d0ea26a37b8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:52.648347 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 537/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:23:52 2022] PUT /identity/v3/projects/48dd7552ee1a49deba21a14cd14dce54/users/67dd81188d6142dca443cb6b855c1aec/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:23:53.440430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a1f70-ddc7-4925-aa29-ccf1ea28fbcc None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:53.440926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a1f70-ddc7-4925-aa29-ccf1ea28fbcc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:53.441049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a1f70-ddc7-4925-aa29-ccf1ea28fbcc None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:53.442343 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 548/1085] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:53.447769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb26ab-a2b1-4857-a86c-1dea76d1d087 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:53.448183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb26ab-a2b1-4857-a86c-1dea76d1d087 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:53.448307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb26ab-a2b1-4857-a86c-1dea76d1d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:53.467918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ffcb26ab-a2b1-4857-a86c-1dea76d1d087 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:53.469084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ffcb26ab-a2b1-4857-a86c-1dea76d1d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:53.540331 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 538/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:53.817798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a1f70-ddc7-4925-aa29-ccf1ea28fbcc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:53.918652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a1f70-ddc7-4925-aa29-ccf1ea28fbcc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:53.933121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:53.959084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b240a6-76f7-4abe-b566-b77a465cda2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:53.959705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b240a6-76f7-4abe-b566-b77a465cda2e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:53.960024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b240a6-76f7-4abe-b566-b77a465cda2e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:53.960393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b240a6-76f7-4abe-b566-b77a465cda2e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:53.971562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b240a6-76f7-4abe-b566-b77a465cda2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a6b240a6-76f7-4abe-b566-b77a465cda2e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:53.976241 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:53.977308 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:53.978337 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 549/1087] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 30 09:23:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:53.983330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb26ab-a2b1-4857-a86c-1dea76d1d087 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:54.074479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:54.083116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb26ab-a2b1-4857-a86c-1dea76d1d087 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:54.116464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:54.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:54.117341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:54.117611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:54.118291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:54.118591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:54.119974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:54.127960 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 539/1088] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 30 09:23:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:54.133821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b240a6-76f7-4abe-b566-b77a465cda2e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:54.232435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b240a6-76f7-4abe-b566-b77a465cda2e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:54.265516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:54.266725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:54.267004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:54.267383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:54.278243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:54.279708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:54.294474 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:54.295486 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:54.296355 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:54.296355 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:54.296355 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:54.296355 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 550/1089] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:54.301352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:54.397658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699d18f-fc80-481f-a6b4-b2f2221f56cf None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:54.431968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:54.432571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:54.432848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:54.433115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:54.433781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:54.434083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:54.435438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:54.446974 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 540/1090] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:54.452587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:54.479786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:54.553565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dbe00-e6eb-49b2-b1b3-dcd025fa6816 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:54.585339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:54.585989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:54.586269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:54.586538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:54.593682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:54.595675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:23:54.606480 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:54.612749 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:54.613812 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 551/1091] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 30 09:23:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:54.614400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:54.632484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:54.717430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30122085-9314-4df6-8040-65a083ba6534 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:54.751807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e209b17-aa05-4b2d-b1ab-e041d3218b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:54.752401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209b17-aa05-4b2d-b1ab-e041d3218b76 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:54.752683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209b17-aa05-4b2d-b1ab-e041d3218b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:54.752954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209b17-aa05-4b2d-b1ab-e041d3218b76 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:54.753706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e209b17-aa05-4b2d-b1ab-e041d3218b76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:54.754012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e209b17-aa05-4b2d-b1ab-e041d3218b76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:54.755435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e209b17-aa05-4b2d-b1ab-e041d3218b76 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:54.772258 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 541/1092] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 30 09:23:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:54.778357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:54.786639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac196-7703-405c-b31e-5ea01a51957e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:54.786924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac196-7703-405c-b31e-5ea01a51957e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:54.787199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac196-7703-405c-b31e-5ea01a51957e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:54.788491 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 542/1093] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:23:54.793596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d07eb0-0f7c-4cb7-8e8a-4076c255413e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:54.793911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d07eb0-0f7c-4cb7-8e8a-4076c255413e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:54.794196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d07eb0-0f7c-4cb7-8e8a-4076c255413e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:54.811997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-48d07eb0-0f7c-4cb7-8e8a-4076c255413e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:54.813788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-48d07eb0-0f7c-4cb7-8e8a-4076c255413e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:54.862933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:54.879556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffff68f-2744-486b-9e9e-d89766d074ca None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:54.882890 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 543/1094] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:54.912585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34446-4224-42de-8363-f27107c87b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:54.913185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34446-4224-42de-8363-f27107c87b1a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:54.913467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34446-4224-42de-8363-f27107c87b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:54.913789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e34446-4224-42de-8363-f27107c87b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:54.980641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e34446-4224-42de-8363-f27107c87b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:54.982050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e34446-4224-42de-8363-f27107c87b1a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.061731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:55.078464 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 552/1095] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 09:23:55.084404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d07eb0-0f7c-4cb7-8e8a-4076c255413e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.087812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:55.154660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34446-4224-42de-8363-f27107c87b1a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.181734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d07eb0-0f7c-4cb7-8e8a-4076c255413e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.215301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.215926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.216203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.216472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.226659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:55.228111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:23:55.242953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:55.243964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:23:55.244876 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:23:55.244876 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:23:55.244876 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:55.244876 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 544/1096] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:55.249916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.256444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e34446-4224-42de-8363-f27107c87b1a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.293933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76b3d6-512a-43fe-8e5e-74700ea4c505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.294806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76b3d6-512a-43fe-8e5e-74700ea4c505 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.295090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76b3d6-512a-43fe-8e5e-74700ea4c505 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.295364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76b3d6-512a-43fe-8e5e-74700ea4c505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.305916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76b3d6-512a-43fe-8e5e-74700ea4c505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:23:55.307471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-3a76b3d6-512a-43fe-8e5e-74700ea4c505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:23:55.310691 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:55.311643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:23:55.312476 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 553/1097] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:23:55.317830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76b3d6-512a-43fe-8e5e-74700ea4c505 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.352087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697fe18-514e-4536-bd22-165eb79558c6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.364567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:55.386467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.387070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.387344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.387615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.388249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:55.388547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:55.389933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.402186 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 545/1098] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:55.409043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.422086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76b3d6-512a-43fe-8e5e-74700ea4c505 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.456999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.457594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.457908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.458182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.458843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:55.459153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:55.460530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.468751 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 554/1099] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:55.474744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.508863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f5c48-a94a-4126-b814-5e11e3e688b0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.525635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:55.543549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.544175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.544458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.544733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.551907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:55.553349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:23:55.565498 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:23:55.566408 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:23:55.567328 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 546/1100] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:23:55.572465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.574981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeeaee9-bbba-4ec8-8005-5ced0a6f515a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.582074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:55.608083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2492b-6b8e-499d-aefe-03054c66be53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.608702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2492b-6b8e-499d-aefe-03054c66be53 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.608979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2492b-6b8e-499d-aefe-03054c66be53 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.609247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2492b-6b8e-499d-aefe-03054c66be53 None admin] PATH_INFO: `/v3/users/67dd81188d6142dca443cb6b855c1aec` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.610971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2492b-6b8e-499d-aefe-03054c66be53 None admin] RBAC: Authorizing `identity:get_user(user_id=67dd81188d6142dca443cb6b855c1aec)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:55.612417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2492b-6b8e-499d-aefe-03054c66be53 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.614023 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 555/1101] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/users/67dd81188d6142dca443cb6b855c1aec => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:55.619695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2492b-6b8e-499d-aefe-03054c66be53 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.669590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b7a44-b265-466d-836b-ffc07a4d1b8d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.701655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.702249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.702528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.702799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.704044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:55.704378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:55.705820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.717694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2492b-6b8e-499d-aefe-03054c66be53 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.718612 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 547/1102] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:55.725452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:55.752211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814511ac-39d1-4545-9671-21221998e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.752810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814511ac-39d1-4545-9671-21221998e7ac None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.753083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814511ac-39d1-4545-9671-21221998e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.753354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814511ac-39d1-4545-9671-21221998e7ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.759862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:55.796553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:55.821692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814511ac-39d1-4545-9671-21221998e7ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:55.823174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814511ac-39d1-4545-9671-21221998e7ac None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.823652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e52d-a9ac-4446-a934-782901034b35 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:55.857747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:55.858345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:55.858617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:55.858882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:55.859589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] RBAC: Adding query filter params (role.id=61285fdf54ab4933b5e0b2f767155362, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=aa055ee122ce4e92b4da7f8222521a9e) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:55.859931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:55.861452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:55.867084 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 548/1103] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/role_assignments?user.id=aa055ee122ce4e92b4da7f8222521a9e&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=61285fdf54ab4933b5e0b2f767155362 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:55.926569 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 556/1104] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 30 09:23:55.932852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:56.035149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:56.038637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1c23d-a3fe-47f7-8bf5-c7384688d61b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:56.073120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11235b0-bf22-4daa-9104-06d135695471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:56.073779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11235b0-bf22-4daa-9104-06d135695471 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:56.074054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11235b0-bf22-4daa-9104-06d135695471 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:56.074327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11235b0-bf22-4daa-9104-06d135695471 None admin] PATH_INFO: `/v3/projects/48dd7552ee1a49deba21a14cd14dce54` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:56.075425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11235b0-bf22-4daa-9104-06d135695471 None admin] RBAC: Authorizing `identity:get_project(project_id=48dd7552ee1a49deba21a14cd14dce54)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:56.076851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11235b0-bf22-4daa-9104-06d135695471 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:56.078544 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 549/1105] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:23:55 2022] GET /identity/v3/projects/48dd7552ee1a49deba21a14cd14dce54 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:23:56.085372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814511ac-39d1-4545-9671-21221998e7ac None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:56.163643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:56.189903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814511ac-39d1-4545-9671-21221998e7ac None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:56.224872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b4963-f528-4687-abb4-77218075ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:56.225472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4963-f528-4687-abb4-77218075ee65 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:56.225782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4963-f528-4687-abb4-77218075ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:56.226053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4963-f528-4687-abb4-77218075ee65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:56.226767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b4963-f528-4687-abb4-77218075ee65 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=48dd7552ee1a49deba21a14cd14dce54, user.id=67dd81188d6142dca443cb6b855c1aec) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:23:56.227073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b4963-f528-4687-abb4-77218075ee65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:56.228504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b4963-f528-4687-abb4-77218075ee65 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:56.234451 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 557/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:23:56 2022] GET /identity/v3/role_assignments?user.id=67dd81188d6142dca443cb6b855c1aec&scope.project.id=48dd7552ee1a49deba21a14cd14dce54&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:58.441672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24404759-bb3a-49dd-a124-67f8afbf9cf6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:58.442138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24404759-bb3a-49dd-a124-67f8afbf9cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:58.442229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24404759-bb3a-49dd-a124-67f8afbf9cf6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:58.443553 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 550/1107] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:23:58.448816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba4f9-043a-4573-87ea-4b6f5dcc0ea5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:58.449232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba4f9-043a-4573-87ea-4b6f5dcc0ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:58.449356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aba4f9-043a-4573-87ea-4b6f5dcc0ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:58.467519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c6aba4f9-043a-4573-87ea-4b6f5dcc0ea5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:23:58.468685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c6aba4f9-043a-4573-87ea-4b6f5dcc0ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:23:58.531448 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 558/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:23:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:23:58.798600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24404759-bb3a-49dd-a124-67f8afbf9cf6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:58.898154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24404759-bb3a-49dd-a124-67f8afbf9cf6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:58.931249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a6088-94af-4c77-8a74-e642e953351a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:58.931864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6088-94af-4c77-8a74-e642e953351a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:58.932142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6088-94af-4c77-8a74-e642e953351a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:58.932414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6088-94af-4c77-8a74-e642e953351a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:58.933504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a6088-94af-4c77-8a74-e642e953351a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:58.934954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a6088-94af-4c77-8a74-e642e953351a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:58.936481 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 551/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:23:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:23:58.942625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba4f9-043a-4573-87ea-4b6f5dcc0ea5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:23:59.006907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:23:59.040351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aba4f9-043a-4573-87ea-4b6f5dcc0ea5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:23:59.073846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69065f6e-395f-4e41-aad3-1356271153ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:23:59.074384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69065f6e-395f-4e41-aad3-1356271153ab None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:23:59.074664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69065f6e-395f-4e41-aad3-1356271153ab None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:23:59.074936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69065f6e-395f-4e41-aad3-1356271153ab None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:23:59.075589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69065f6e-395f-4e41-aad3-1356271153ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:23:59.077024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69065f6e-395f-4e41-aad3-1356271153ab None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:23:59.100098 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 559/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:23:58 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:24:01.299173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880340-64f7-41af-b169-57fae487c7e5 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:01.299705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880340-64f7-41af-b169-57fae487c7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:01.299705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880340-64f7-41af-b169-57fae487c7e5 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:01.301041 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 552/1111] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:01.306367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0912b7-a0df-456e-b86c-1ad63a826e40 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:01.306785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0912b7-a0df-456e-b86c-1ad63a826e40 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:01.306913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0912b7-a0df-456e-b86c-1ad63a826e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:01.325124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0d0912b7-a0df-456e-b86c-1ad63a826e40 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:01.326323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0d0912b7-a0df-456e-b86c-1ad63a826e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:01.391706 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 560/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:24:01.670281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40880340-64f7-41af-b169-57fae487c7e5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:01.706266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:01.775935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40880340-64f7-41af-b169-57fae487c7e5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:01.810779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b20d75-fa8f-4195-88de-e9ccd5d984f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:01.811395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b20d75-fa8f-4195-88de-e9ccd5d984f6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:01.811674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b20d75-fa8f-4195-88de-e9ccd5d984f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:01.811948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b20d75-fa8f-4195-88de-e9ccd5d984f6 None admin] PATH_INFO: `/v3/roles/89a7fdb1750447479ee3540579dc1e23` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:01.813998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b20d75-fa8f-4195-88de-e9ccd5d984f6 None admin] RBAC: Authorizing `identity:get_role(role_id=89a7fdb1750447479ee3540579dc1e23)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:01.815369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b20d75-fa8f-4195-88de-e9ccd5d984f6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:01.816675 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 553/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:01 2022] GET /identity/v3/roles/89a7fdb1750447479ee3540579dc1e23 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:01.822550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0912b7-a0df-456e-b86c-1ad63a826e40 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:01.875684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:01.920263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0912b7-a0df-456e-b86c-1ad63a826e40 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:01.953086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520c1ff-e83b-4c48-a7db-7899d1c53f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:01.953840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c1ff-e83b-4c48-a7db-7899d1c53f28 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:01.954102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c1ff-e83b-4c48-a7db-7899d1c53f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:01.954379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c1ff-e83b-4c48-a7db-7899d1c53f28 None admin] PATH_INFO: `/v3/users/fb4030299dbc4ad69254c169898bcd6b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:01.966968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520c1ff-e83b-4c48-a7db-7899d1c53f28 None admin] RBAC: Authorizing `identity:get_user(user_id=fb4030299dbc4ad69254c169898bcd6b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:01.968452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520c1ff-e83b-4c48-a7db-7899d1c53f28 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:01.970616 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 561/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:01 2022] GET /identity/v3/users/fb4030299dbc4ad69254c169898bcd6b => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:01.976397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b20d75-fa8f-4195-88de-e9ccd5d984f6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:02.073113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b20d75-fa8f-4195-88de-e9ccd5d984f6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:02.075387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:02.105600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7148632-8a0c-4bfa-8aad-c76050280d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:02.106224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148632-8a0c-4bfa-8aad-c76050280d4d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:02.106501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148632-8a0c-4bfa-8aad-c76050280d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:02.106774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148632-8a0c-4bfa-8aad-c76050280d4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:02.171033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7148632-8a0c-4bfa-8aad-c76050280d4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:02.173268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7148632-8a0c-4bfa-8aad-c76050280d4d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:02.267737 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 554/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:24:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:24:02.273582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520c1ff-e83b-4c48-a7db-7899d1c53f28 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:02.372904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520c1ff-e83b-4c48-a7db-7899d1c53f28 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:02.387028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:02.411166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da237736-bfcc-4d53-b36b-f78602378610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:02.411798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da237736-bfcc-4d53-b36b-f78602378610 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:02.412076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da237736-bfcc-4d53-b36b-f78602378610 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:02.412348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da237736-bfcc-4d53-b36b-f78602378610 None admin] PATH_INFO: `/v3/projects/48dd7552ee1a49deba21a14cd14dce54` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:02.413470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da237736-bfcc-4d53-b36b-f78602378610 None admin] RBAC: Authorizing `identity:get_project(project_id=48dd7552ee1a49deba21a14cd14dce54)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:02.414937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da237736-bfcc-4d53-b36b-f78602378610 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:02.416557 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 562/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:02 2022] GET /identity/v3/projects/48dd7552ee1a49deba21a14cd14dce54 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:02.422841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7148632-8a0c-4bfa-8aad-c76050280d4d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:02.425872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:02.516886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7148632-8a0c-4bfa-8aad-c76050280d4d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:02.549323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ac8a9-78a2-4670-a4b6-53f78d8c9340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:02.549869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac8a9-78a2-4670-a4b6-53f78d8c9340 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:02.550145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac8a9-78a2-4670-a4b6-53f78d8c9340 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:02.550411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ac8a9-78a2-4670-a4b6-53f78d8c9340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:02.551117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ac8a9-78a2-4670-a4b6-53f78d8c9340 None admin] RBAC: Adding query filter params (role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=48dd7552ee1a49deba21a14cd14dce54, user.id=fb4030299dbc4ad69254c169898bcd6b) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:02.551418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ac8a9-78a2-4670-a4b6-53f78d8c9340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:02.552831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ac8a9-78a2-4670-a4b6-53f78d8c9340 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:02.557957 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 555/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:24:02 2022] GET /identity/v3/role_assignments?user.id=fb4030299dbc4ad69254c169898bcd6b&scope.project.id=48dd7552ee1a49deba21a14cd14dce54&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:04.665929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3de0d-61f2-4014-b6c8-f4d80c17c686 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:04.666424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3de0d-61f2-4014-b6c8-f4d80c17c686 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:04.666424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3de0d-61f2-4014-b6c8-f4d80c17c686 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:04.667725 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 563/1118] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:04.673561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bdfee-ca3c-4a71-87fd-866f4e7790eb None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:04.673968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bdfee-ca3c-4a71-87fd-866f4e7790eb None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:04.674137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bdfee-ca3c-4a71-87fd-866f4e7790eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:04.692594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9e6bdfee-ca3c-4a71-87fd-866f4e7790eb None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:04.693820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9e6bdfee-ca3c-4a71-87fd-866f4e7790eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:04.753120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:04.758979 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 556/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:05.033450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3de0d-61f2-4014-b6c8-f4d80c17c686 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:05.138300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3de0d-61f2-4014-b6c8-f4d80c17c686 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:05.172553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124756d5-dff7-4906-aa33-b2e8eb9a48be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:05.173160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124756d5-dff7-4906-aa33-b2e8eb9a48be None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:05.173437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124756d5-dff7-4906-aa33-b2e8eb9a48be None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:05.173728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124756d5-dff7-4906-aa33-b2e8eb9a48be None admin] PATH_INFO: `/v3/roles/89a7fdb1750447479ee3540579dc1e23` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:05.175096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124756d5-dff7-4906-aa33-b2e8eb9a48be None admin] RBAC: Authorizing `identity:get_role(role_id=89a7fdb1750447479ee3540579dc1e23)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:05.176460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124756d5-dff7-4906-aa33-b2e8eb9a48be None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:05.177790 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 564/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:05 2022] GET /identity/v3/roles/89a7fdb1750447479ee3540579dc1e23 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:05.183661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bdfee-ca3c-4a71-87fd-866f4e7790eb None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:05.278414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bdfee-ca3c-4a71-87fd-866f4e7790eb None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:05.311414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03accfdd-b101-4304-b906-5dd97898ec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:05.311975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03accfdd-b101-4304-b906-5dd97898ec0c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:05.312258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03accfdd-b101-4304-b906-5dd97898ec0c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:05.313190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03accfdd-b101-4304-b906-5dd97898ec0c None admin] PATH_INFO: `/v3/users/fb4030299dbc4ad69254c169898bcd6b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:05.314336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03accfdd-b101-4304-b906-5dd97898ec0c None admin] RBAC: Authorizing `identity:get_user(user_id=fb4030299dbc4ad69254c169898bcd6b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:05.315766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03accfdd-b101-4304-b906-5dd97898ec0c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:05.317323 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 557/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:05 2022] GET /identity/v3/users/fb4030299dbc4ad69254c169898bcd6b => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:05.322963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124756d5-dff7-4906-aa33-b2e8eb9a48be None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:05.370465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:05.419876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124756d5-dff7-4906-aa33-b2e8eb9a48be None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:05.452035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40678d04-612c-4b8c-96b1-1beaf2e6eb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:05.452645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678d04-612c-4b8c-96b1-1beaf2e6eb8e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:05.452922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678d04-612c-4b8c-96b1-1beaf2e6eb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:05.457816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40678d04-612c-4b8c-96b1-1beaf2e6eb8e None admin] PATH_INFO: `/v3/projects/48dd7552ee1a49deba21a14cd14dce54` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:05.458932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40678d04-612c-4b8c-96b1-1beaf2e6eb8e None admin] RBAC: Authorizing `identity:get_project(project_id=48dd7552ee1a49deba21a14cd14dce54)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:05.460360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40678d04-612c-4b8c-96b1-1beaf2e6eb8e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:05.461987 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 565/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:05 2022] GET /identity/v3/projects/48dd7552ee1a49deba21a14cd14dce54 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:05.467871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03accfdd-b101-4304-b906-5dd97898ec0c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:05.535586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:05.569020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03accfdd-b101-4304-b906-5dd97898ec0c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:05.601974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8072883-827c-4ef0-8ab5-22fba8d151cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:05.602623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072883-827c-4ef0-8ab5-22fba8d151cc None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:05.602904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072883-827c-4ef0-8ab5-22fba8d151cc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:05.603180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8072883-827c-4ef0-8ab5-22fba8d151cc None admin] PATH_INFO: `/v3/projects/48dd7552ee1a49deba21a14cd14dce54/users/fb4030299dbc4ad69254c169898bcd6b/roles/89a7fdb1750447479ee3540579dc1e23` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:05.605306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8072883-827c-4ef0-8ab5-22fba8d151cc None admin] RBAC: Authorizing `identity:create_grant(project_id=48dd7552ee1a49deba21a14cd14dce54, user_id=fb4030299dbc4ad69254c169898bcd6b, role_id=89a7fdb1750447479ee3540579dc1e23)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:05.606761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8072883-827c-4ef0-8ab5-22fba8d151cc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:05.618726 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 558/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:24:05 2022] PUT /identity/v3/projects/48dd7552ee1a49deba21a14cd14dce54/users/fb4030299dbc4ad69254c169898bcd6b/roles/89a7fdb1750447479ee3540579dc1e23 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:24:07.761781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460039fc-d3c1-4f34-beb0-95897b3cb31d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:07.762303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460039fc-d3c1-4f34-beb0-95897b3cb31d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:07.762303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460039fc-d3c1-4f34-beb0-95897b3cb31d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:07.763609 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 566/1124] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:07.770236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52942b-60bb-4f35-8497-c16288fb5c67 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:07.770618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52942b-60bb-4f35-8497-c16288fb5c67 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:07.770883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52942b-60bb-4f35-8497-c16288fb5c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:07.790415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1e52942b-60bb-4f35-8497-c16288fb5c67 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:07.791609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1e52942b-60bb-4f35-8497-c16288fb5c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:07.856335 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 559/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:08.125315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460039fc-d3c1-4f34-beb0-95897b3cb31d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:08.224412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460039fc-d3c1-4f34-beb0-95897b3cb31d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:08.240658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:08.259164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870dd2e-5422-4ff6-a188-caa4bbbbeed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:08.259785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870dd2e-5422-4ff6-a188-caa4bbbbeed7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:08.260063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870dd2e-5422-4ff6-a188-caa4bbbbeed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:08.260333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870dd2e-5422-4ff6-a188-caa4bbbbeed7 None admin] PATH_INFO: `/v3/roles/89a7fdb1750447479ee3540579dc1e23` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:08.261747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870dd2e-5422-4ff6-a188-caa4bbbbeed7 None admin] RBAC: Authorizing `identity:get_role(role_id=89a7fdb1750447479ee3540579dc1e23)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:08.263125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870dd2e-5422-4ff6-a188-caa4bbbbeed7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:08.264432 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 567/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:08 2022] GET /identity/v3/roles/89a7fdb1750447479ee3540579dc1e23 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:08.270300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52942b-60bb-4f35-8497-c16288fb5c67 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:08.324054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:08.366640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52942b-60bb-4f35-8497-c16288fb5c67 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:08.399527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5dcc6-2a65-46d0-9955-bb3b444a7214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:08.400360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5dcc6-2a65-46d0-9955-bb3b444a7214 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:08.400639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5dcc6-2a65-46d0-9955-bb3b444a7214 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:08.400915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5dcc6-2a65-46d0-9955-bb3b444a7214 None admin] PATH_INFO: `/v3/users/fb4030299dbc4ad69254c169898bcd6b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:08.402064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5dcc6-2a65-46d0-9955-bb3b444a7214 None admin] RBAC: Authorizing `identity:get_user(user_id=fb4030299dbc4ad69254c169898bcd6b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:08.403502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5dcc6-2a65-46d0-9955-bb3b444a7214 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:08.405096 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 560/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:08 2022] GET /identity/v3/users/fb4030299dbc4ad69254c169898bcd6b => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:08.411204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870dd2e-5422-4ff6-a188-caa4bbbbeed7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:08.508548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870dd2e-5422-4ff6-a188-caa4bbbbeed7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:08.540487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c3e01-0956-495e-a39e-1850f3d102db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:08.541072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c3e01-0956-495e-a39e-1850f3d102db None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:08.541350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c3e01-0956-495e-a39e-1850f3d102db None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:08.541643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c3e01-0956-495e-a39e-1850f3d102db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:08.591307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:08.607126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c3e01-0956-495e-a39e-1850f3d102db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:08.608567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c3e01-0956-495e-a39e-1850f3d102db None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:08.704021 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 568/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:24:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:24:08.709986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5dcc6-2a65-46d0-9955-bb3b444a7214 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:08.803951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5dcc6-2a65-46d0-9955-bb3b444a7214 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:08.826375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:08.837449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130e082-3dc7-430c-b4fe-a64cf822f3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:08.838105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130e082-3dc7-430c-b4fe-a64cf822f3fd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:08.838385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130e082-3dc7-430c-b4fe-a64cf822f3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:08.838657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130e082-3dc7-430c-b4fe-a64cf822f3fd None admin] PATH_INFO: `/v3/projects/48dd7552ee1a49deba21a14cd14dce54` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:08.839757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130e082-3dc7-430c-b4fe-a64cf822f3fd None admin] RBAC: Authorizing `identity:get_project(project_id=48dd7552ee1a49deba21a14cd14dce54)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:08.841173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130e082-3dc7-430c-b4fe-a64cf822f3fd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:08.842808 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 561/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:08 2022] GET /identity/v3/projects/48dd7552ee1a49deba21a14cd14dce54 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:08.849154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c3e01-0956-495e-a39e-1850f3d102db None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:08.929086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:08.945323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c3e01-0956-495e-a39e-1850f3d102db None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:08.978656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4424f90-9b5c-4140-8558-172749973c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:08.979183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424f90-9b5c-4140-8558-172749973c39 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:08.979459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424f90-9b5c-4140-8558-172749973c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:08.979727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4424f90-9b5c-4140-8558-172749973c39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:08.980427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4424f90-9b5c-4140-8558-172749973c39 None admin] RBAC: Adding query filter params (role.id=89a7fdb1750447479ee3540579dc1e23, scope.project.id=48dd7552ee1a49deba21a14cd14dce54, user.id=fb4030299dbc4ad69254c169898bcd6b) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:08.980730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4424f90-9b5c-4140-8558-172749973c39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:08.982188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4424f90-9b5c-4140-8558-172749973c39 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:08.988430 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 569/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:24:08 2022] GET /identity/v3/role_assignments?user.id=fb4030299dbc4ad69254c169898bcd6b&scope.project.id=48dd7552ee1a49deba21a14cd14dce54&role.id=89a7fdb1750447479ee3540579dc1e23 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:11.133073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60f324-af21-48ac-a8ab-0a4a3cc2cba5 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:11.133621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60f324-af21-48ac-a8ab-0a4a3cc2cba5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:11.133621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60f324-af21-48ac-a8ab-0a4a3cc2cba5 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:11.134993 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 562/1131] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:11.141059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58797dd2-d10a-4879-b095-43f7ef4b3681 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:11.141521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58797dd2-d10a-4879-b095-43f7ef4b3681 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:11.141618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58797dd2-d10a-4879-b095-43f7ef4b3681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:11.159511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-58797dd2-d10a-4879-b095-43f7ef4b3681 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:11.160686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-58797dd2-d10a-4879-b095-43f7ef4b3681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:11.226538 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 570/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:11.504039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60f324-af21-48ac-a8ab-0a4a3cc2cba5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:11.602393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60f324-af21-48ac-a8ab-0a4a3cc2cba5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:11.635009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef540045-87be-46f6-af4f-373569feaa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:11.635624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef540045-87be-46f6-af4f-373569feaa42 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:11.635900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef540045-87be-46f6-af4f-373569feaa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:11.636172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef540045-87be-46f6-af4f-373569feaa42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:11.637268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef540045-87be-46f6-af4f-373569feaa42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:11.638723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef540045-87be-46f6-af4f-373569feaa42 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:11.640374 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 563/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:24:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:11.646365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58797dd2-d10a-4879-b095-43f7ef4b3681 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:11.721707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:11.742709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58797dd2-d10a-4879-b095-43f7ef4b3681 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:11.775353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73597061-61e0-4c97-bcfe-af0fd6e141b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:11.775951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73597061-61e0-4c97-bcfe-af0fd6e141b6 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:11.776227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73597061-61e0-4c97-bcfe-af0fd6e141b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:11.776493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73597061-61e0-4c97-bcfe-af0fd6e141b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:11.777136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73597061-61e0-4c97-bcfe-af0fd6e141b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:11.778594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73597061-61e0-4c97-bcfe-af0fd6e141b6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:11.806017 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 571/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:24:11 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:24:14.000738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe31-ed64-48e1-99e0-3d56351b8d90 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:14.001287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe31-ed64-48e1-99e0-3d56351b8d90 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:14.001287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afe31-ed64-48e1-99e0-3d56351b8d90 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:14.002655 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 564/1135] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:14.007982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4acf6-cbdc-4672-bd95-4d1ce370ab70 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:14.008349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4acf6-cbdc-4672-bd95-4d1ce370ab70 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:14.008535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4acf6-cbdc-4672-bd95-4d1ce370ab70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:14.027095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-35e4acf6-cbdc-4672-bd95-4d1ce370ab70 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:14.028275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-35e4acf6-cbdc-4672-bd95-4d1ce370ab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:14.092069 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 572/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:14.364598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afe31-ed64-48e1-99e0-3d56351b8d90 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:14.424065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:14.464361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afe31-ed64-48e1-99e0-3d56351b8d90 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:14.496717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81d512-7188-4530-b00e-2785061a7ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:14.497333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81d512-7188-4530-b00e-2785061a7ade None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:14.497610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81d512-7188-4530-b00e-2785061a7ade None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:14.497909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81d512-7188-4530-b00e-2785061a7ade None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:14.499002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81d512-7188-4530-b00e-2785061a7ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:14.500617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81d512-7188-4530-b00e-2785061a7ade None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:14.502196 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 565/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:24:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:14.507996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4acf6-cbdc-4672-bd95-4d1ce370ab70 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:14.565524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:14.605512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4acf6-cbdc-4672-bd95-4d1ce370ab70 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:14.638293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e87ce2-b4a8-4b4c-8f84-e1e8f16a7cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:14.638881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e87ce2-b4a8-4b4c-8f84-e1e8f16a7cdd None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:14.639156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e87ce2-b4a8-4b4c-8f84-e1e8f16a7cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:14.639424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e87ce2-b4a8-4b4c-8f84-e1e8f16a7cdd None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:14.640129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e87ce2-b4a8-4b4c-8f84-e1e8f16a7cdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:14.642234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e87ce2-b4a8-4b4c-8f84-e1e8f16a7cdd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:14.663250 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 573/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:24:14 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:24:16.862013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db5b40-0ad2-4f5f-b791-139f4d3559f4 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:16.862531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db5b40-0ad2-4f5f-b791-139f4d3559f4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:16.862531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db5b40-0ad2-4f5f-b791-139f4d3559f4 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:16.863784 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 566/1139] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:16.868970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee094b14-c439-480a-95cb-51d9909ee7f1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:16.869363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee094b14-c439-480a-95cb-51d9909ee7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:16.869520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee094b14-c439-480a-95cb-51d9909ee7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:16.887772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ee094b14-c439-480a-95cb-51d9909ee7f1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:16.888949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ee094b14-c439-480a-95cb-51d9909ee7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:16.956120 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 574/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:17.222673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db5b40-0ad2-4f5f-b791-139f4d3559f4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:17.321375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db5b40-0ad2-4f5f-b791-139f4d3559f4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:17.346488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:17.354220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d073735-e3c0-46e4-9017-52f1d3b7a97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:17.354841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d073735-e3c0-46e4-9017-52f1d3b7a97b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:17.355120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d073735-e3c0-46e4-9017-52f1d3b7a97b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:17.355394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d073735-e3c0-46e4-9017-52f1d3b7a97b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:17.365597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d073735-e3c0-46e4-9017-52f1d3b7a97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-4d073735-e3c0-46e4-9017-52f1d3b7a97b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:24:17.370367 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:24:17.371417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:24:17.372271 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 567/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:24:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:24:17.378154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee094b14-c439-480a-95cb-51d9909ee7f1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:17.424294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:17.475532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee094b14-c439-480a-95cb-51d9909ee7f1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:17.508286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:17.508879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:17.509155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:17.509424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:17.510102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:17.510401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:17.511761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:17.519396 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 575/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:24:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:17.525650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d073735-e3c0-46e4-9017-52f1d3b7a97b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:17.621970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d073735-e3c0-46e4-9017-52f1d3b7a97b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:17.656145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0dcef-f202-4610-9690-81c618774247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:17.656785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0dcef-f202-4610-9690-81c618774247 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:17.657067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0dcef-f202-4610-9690-81c618774247 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:17.657343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0dcef-f202-4610-9690-81c618774247 None admin] PATH_INFO: `/v3/users/1dc5673237f64763a0b799dcf2e8db04` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:17.669713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0dcef-f202-4610-9690-81c618774247 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc5673237f64763a0b799dcf2e8db04)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:17.671154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0dcef-f202-4610-9690-81c618774247 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:17.673012 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 568/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:17 2022] GET /identity/v3/users/1dc5673237f64763a0b799dcf2e8db04 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:17.678696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:17.776494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48565e91-2ccd-4078-8065-2e91089374b4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:17.785325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:17.812233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7372e-1b30-4f4d-b7c4-8712d5440be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:17.813039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7372e-1b30-4f4d-b7c4-8712d5440be5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:17.813537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7372e-1b30-4f4d-b7c4-8712d5440be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:17.813974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7372e-1b30-4f4d-b7c4-8712d5440be5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:17.881745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7372e-1b30-4f4d-b7c4-8712d5440be5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:17.883209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7372e-1b30-4f4d-b7c4-8712d5440be5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:17.978501 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 576/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:24:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:24:17.984741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0dcef-f202-4610-9690-81c618774247 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:18.029467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:18.081223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0dcef-f202-4610-9690-81c618774247 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:18.113959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee379736-e4c3-47a8-a3c6-34b27412cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:18.114576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee379736-e4c3-47a8-a3c6-34b27412cb97 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:18.114856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee379736-e4c3-47a8-a3c6-34b27412cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:18.115130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee379736-e4c3-47a8-a3c6-34b27412cb97 None admin] PATH_INFO: `/v3/projects/681e1e0a7acf4ca2ab72ae4e0adbba1f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:18.116243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee379736-e4c3-47a8-a3c6-34b27412cb97 None admin] RBAC: Authorizing `identity:get_project(project_id=681e1e0a7acf4ca2ab72ae4e0adbba1f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:18.117697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee379736-e4c3-47a8-a3c6-34b27412cb97 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:18.119313 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 569/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:17 2022] GET /identity/v3/projects/681e1e0a7acf4ca2ab72ae4e0adbba1f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:18.125466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7372e-1b30-4f4d-b7c4-8712d5440be5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:18.135356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:18.229459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7372e-1b30-4f4d-b7c4-8712d5440be5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:18.262707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e054d82-f539-445a-ba00-5acdc7b8f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:18.263299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e054d82-f539-445a-ba00-5acdc7b8f63a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:18.263577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e054d82-f539-445a-ba00-5acdc7b8f63a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:18.263848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e054d82-f539-445a-ba00-5acdc7b8f63a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:18.264564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e054d82-f539-445a-ba00-5acdc7b8f63a None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=681e1e0a7acf4ca2ab72ae4e0adbba1f, user.id=1dc5673237f64763a0b799dcf2e8db04) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:18.264872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e054d82-f539-445a-ba00-5acdc7b8f63a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:18.266364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e054d82-f539-445a-ba00-5acdc7b8f63a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:18.272332 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 577/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:24:18 2022] GET /identity/v3/role_assignments?user.id=1dc5673237f64763a0b799dcf2e8db04&scope.project.id=681e1e0a7acf4ca2ab72ae4e0adbba1f&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:20.510200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac8ce-5ca0-4cd9-8275-bca7f707ae41 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:20.510781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac8ce-5ca0-4cd9-8275-bca7f707ae41 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:20.510781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac8ce-5ca0-4cd9-8275-bca7f707ae41 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:20.512257 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 570/1147] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:20.517665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3e7c-c59f-4997-81b2-7cdf965687ff None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:20.518129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3e7c-c59f-4997-81b2-7cdf965687ff None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:20.518231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3e7c-c59f-4997-81b2-7cdf965687ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:20.544621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-42aa3e7c-c59f-4997-81b2-7cdf965687ff None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:20.545820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-42aa3e7c-c59f-4997-81b2-7cdf965687ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:20.613719 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 578/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:20.891239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ac8ce-5ca0-4cd9-8275-bca7f707ae41 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:20.989705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ac8ce-5ca0-4cd9-8275-bca7f707ae41 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:21.002923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:21.022848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f622c48-7ab1-4815-a73e-9395c29136b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:21.023466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f622c48-7ab1-4815-a73e-9395c29136b7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:21.023747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f622c48-7ab1-4815-a73e-9395c29136b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:21.024021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f622c48-7ab1-4815-a73e-9395c29136b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:21.034620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f622c48-7ab1-4815-a73e-9395c29136b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3f622c48-7ab1-4815-a73e-9395c29136b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:24:21.039416 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:24:21.040569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:24:21.041653 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 571/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:24:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:24:21.046630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3e7c-c59f-4997-81b2-7cdf965687ff None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:21.048783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:21.143398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3e7c-c59f-4997-81b2-7cdf965687ff None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:21.176144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:21.176735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:21.177269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:21.177544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:21.178222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:21.178519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:21.179880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:21.187566 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 579/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:24:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:21.193253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f622c48-7ab1-4815-a73e-9395c29136b7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:21.288520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f622c48-7ab1-4815-a73e-9395c29136b7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:21.320067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5f660-2be6-455e-b3c1-74678d154fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:21.320671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5f660-2be6-455e-b3c1-74678d154fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:21.320945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5f660-2be6-455e-b3c1-74678d154fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:21.321216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5f660-2be6-455e-b3c1-74678d154fb7 None admin] PATH_INFO: `/v3/users/1dc5673237f64763a0b799dcf2e8db04` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:21.322328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5f660-2be6-455e-b3c1-74678d154fb7 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc5673237f64763a0b799dcf2e8db04)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:21.323742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5f660-2be6-455e-b3c1-74678d154fb7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:21.325263 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 572/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:21 2022] GET /identity/v3/users/1dc5673237f64763a0b799dcf2e8db04 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:21.331562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:21.391406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:21.428693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7413-ff9e-44aa-8fb6-b8680e7adf07 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:21.461102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a4e38-f44a-4054-9910-ec382dad256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:21.461746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a4e38-f44a-4054-9910-ec382dad256b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:21.462024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a4e38-f44a-4054-9910-ec382dad256b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:21.462291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a4e38-f44a-4054-9910-ec382dad256b None admin] PATH_INFO: `/v3/projects/681e1e0a7acf4ca2ab72ae4e0adbba1f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:21.463387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a4e38-f44a-4054-9910-ec382dad256b None admin] RBAC: Authorizing `identity:get_project(project_id=681e1e0a7acf4ca2ab72ae4e0adbba1f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:21.464809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a4e38-f44a-4054-9910-ec382dad256b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:21.466429 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 580/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:21 2022] GET /identity/v3/projects/681e1e0a7acf4ca2ab72ae4e0adbba1f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:21.473146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5f660-2be6-455e-b3c1-74678d154fb7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:21.516576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:21.571510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5f660-2be6-455e-b3c1-74678d154fb7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:21.603859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfedf3-b817-4df8-b5e4-06ebdce150dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:21.604507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfedf3-b817-4df8-b5e4-06ebdce150dd None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:21.604789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfedf3-b817-4df8-b5e4-06ebdce150dd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:21.605066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfedf3-b817-4df8-b5e4-06ebdce150dd None admin] PATH_INFO: `/v3/projects/681e1e0a7acf4ca2ab72ae4e0adbba1f/users/1dc5673237f64763a0b799dcf2e8db04/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:21.606978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfedf3-b817-4df8-b5e4-06ebdce150dd None admin] RBAC: Authorizing `identity:create_grant(project_id=681e1e0a7acf4ca2ab72ae4e0adbba1f, user_id=1dc5673237f64763a0b799dcf2e8db04, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:21.608404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfedf3-b817-4df8-b5e4-06ebdce150dd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:21.621332 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 573/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:24:21 2022] PUT /identity/v3/projects/681e1e0a7acf4ca2ab72ae4e0adbba1f/users/1dc5673237f64763a0b799dcf2e8db04/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:24:23.763392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce78de3-0fbe-4021-9b21-3da719cae740 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:23.763907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce78de3-0fbe-4021-9b21-3da719cae740 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:23.763907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce78de3-0fbe-4021-9b21-3da719cae740 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:23.765802 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 581/1154] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:24:23.771195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4079c5-f852-4da5-9bc4-d3d2804d54bf None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:23.771636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4079c5-f852-4da5-9bc4-d3d2804d54bf None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:23.771731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4079c5-f852-4da5-9bc4-d3d2804d54bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:23.791090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8a4079c5-f852-4da5-9bc4-d3d2804d54bf None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:23.792281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8a4079c5-f852-4da5-9bc4-d3d2804d54bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:23.856678 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 574/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:24.133354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce78de3-0fbe-4021-9b21-3da719cae740 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:24.232243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce78de3-0fbe-4021-9b21-3da719cae740 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:24.261484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:24.266952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6c2d8-513e-4859-9aae-3af25a57a076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:24.267568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c2d8-513e-4859-9aae-3af25a57a076 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:24.267848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c2d8-513e-4859-9aae-3af25a57a076 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:24.268120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6c2d8-513e-4859-9aae-3af25a57a076 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:24.278227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6c2d8-513e-4859-9aae-3af25a57a076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c1e6c2d8-513e-4859-9aae-3af25a57a076 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:24:24.282990 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:24:24.284080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:24:24.285081 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 582/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:24:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:24:24.290459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4079c5-f852-4da5-9bc4-d3d2804d54bf None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:24.320919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:24.387616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4079c5-f852-4da5-9bc4-d3d2804d54bf None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:24.423137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:24.423740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:24.424015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:24.424285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:24.424939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:24.425238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:24.426628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:24.434320 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 575/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:24:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:24.440253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6c2d8-513e-4859-9aae-3af25a57a076 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:24.537048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6c2d8-513e-4859-9aae-3af25a57a076 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:24.570000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27028-5ce0-4dca-8352-961ae2888cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:24.570617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27028-5ce0-4dca-8352-961ae2888cdc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:24.570900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27028-5ce0-4dca-8352-961ae2888cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:24.571174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27028-5ce0-4dca-8352-961ae2888cdc None admin] PATH_INFO: `/v3/users/1dc5673237f64763a0b799dcf2e8db04` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:24.572279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c27028-5ce0-4dca-8352-961ae2888cdc None admin] RBAC: Authorizing `identity:get_user(user_id=1dc5673237f64763a0b799dcf2e8db04)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:24.573745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c27028-5ce0-4dca-8352-961ae2888cdc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:24.575317 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 583/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:24 2022] GET /identity/v3/users/1dc5673237f64763a0b799dcf2e8db04 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:24.581006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:24.670269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:24.677566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14c483-9b3c-4d88-ad62-5c343ae2d1c7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:24.709460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c4db2-0d2b-4758-acbd-d0f8c41d6f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:24.710082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c4db2-0d2b-4758-acbd-d0f8c41d6f01 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:24.710357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c4db2-0d2b-4758-acbd-d0f8c41d6f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:24.710624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c4db2-0d2b-4758-acbd-d0f8c41d6f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:24.773314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c4db2-0d2b-4758-acbd-d0f8c41d6f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:24.775014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c4db2-0d2b-4758-acbd-d0f8c41d6f01 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:24.859972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:24.867222 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 576/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:24:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:24:24.873176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27028-5ce0-4dca-8352-961ae2888cdc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:24.931417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:24.979476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27028-5ce0-4dca-8352-961ae2888cdc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:25.011954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e99169-65a9-42e9-9ce6-8f0ba2e217cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:25.012580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99169-65a9-42e9-9ce6-8f0ba2e217cc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:25.012863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99169-65a9-42e9-9ce6-8f0ba2e217cc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:25.013141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99169-65a9-42e9-9ce6-8f0ba2e217cc None admin] PATH_INFO: `/v3/projects/681e1e0a7acf4ca2ab72ae4e0adbba1f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:25.014259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e99169-65a9-42e9-9ce6-8f0ba2e217cc None admin] RBAC: Authorizing `identity:get_project(project_id=681e1e0a7acf4ca2ab72ae4e0adbba1f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:25.016246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e99169-65a9-42e9-9ce6-8f0ba2e217cc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:25.017911 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 584/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:24 2022] GET /identity/v3/projects/681e1e0a7acf4ca2ab72ae4e0adbba1f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:25.024052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c4db2-0d2b-4758-acbd-d0f8c41d6f01 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:25.120574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c4db2-0d2b-4758-acbd-d0f8c41d6f01 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:25.153166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098f1c3-b96b-4521-8671-91416609ed85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:25.153787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098f1c3-b96b-4521-8671-91416609ed85 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:25.154063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098f1c3-b96b-4521-8671-91416609ed85 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:25.154330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098f1c3-b96b-4521-8671-91416609ed85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:25.155034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098f1c3-b96b-4521-8671-91416609ed85 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=681e1e0a7acf4ca2ab72ae4e0adbba1f, user.id=1dc5673237f64763a0b799dcf2e8db04) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:25.155438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098f1c3-b96b-4521-8671-91416609ed85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:25.156851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098f1c3-b96b-4521-8671-91416609ed85 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:25.162313 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 577/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:24:25 2022] GET /identity/v3/role_assignments?user.id=1dc5673237f64763a0b799dcf2e8db04&scope.project.id=681e1e0a7acf4ca2ab72ae4e0adbba1f&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:27.300532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9ab9a-8ece-46db-9d67-c85a49e159d7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:27.301068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9ab9a-8ece-46db-9d67-c85a49e159d7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:27.301068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b9ab9a-8ece-46db-9d67-c85a49e159d7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:27.302448 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 585/1162] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:27.307696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eccd14-c101-4902-98c2-fa6cc7b1ed48 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:27.308120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eccd14-c101-4902-98c2-fa6cc7b1ed48 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:27.308244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eccd14-c101-4902-98c2-fa6cc7b1ed48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:27.326421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b4eccd14-c101-4902-98c2-fa6cc7b1ed48 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:27.328010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b4eccd14-c101-4902-98c2-fa6cc7b1ed48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:27.382394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:27.398852 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 578/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:27.668417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9ab9a-8ece-46db-9d67-c85a49e159d7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:27.768100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b9ab9a-8ece-46db-9d67-c85a49e159d7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:27.793298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:27.801780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68c6d4-ac27-49d6-b9d3-ad31df58e212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:27.802378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c6d4-ac27-49d6-b9d3-ad31df58e212 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:27.802658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c6d4-ac27-49d6-b9d3-ad31df58e212 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:27.802930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68c6d4-ac27-49d6-b9d3-ad31df58e212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:27.870232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68c6d4-ac27-49d6-b9d3-ad31df58e212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:27.871704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68c6d4-ac27-49d6-b9d3-ad31df58e212 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:27.967450 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 586/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:24:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:24:27.973320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eccd14-c101-4902-98c2-fa6cc7b1ed48 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:28.069315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eccd14-c101-4902-98c2-fa6cc7b1ed48 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:28.101758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d100-db04-4852-988b-db3e4d1c4ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:28.102378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d100-db04-4852-988b-db3e4d1c4ecf None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:28.102658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d100-db04-4852-988b-db3e4d1c4ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:28.102929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d100-db04-4852-988b-db3e4d1c4ecf None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:28.109931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d100-db04-4852-988b-db3e4d1c4ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:28.111375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d100-db04-4852-988b-db3e4d1c4ecf None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3689d100-db04-4852-988b-db3e4d1c4ecf None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:24:28.122811 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:24:28.123976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:24:28.125089 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 579/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:24:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:24:28.129950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68c6d4-ac27-49d6-b9d3-ad31df58e212 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:28.157233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:28.235848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68c6d4-ac27-49d6-b9d3-ad31df58e212 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:28.269909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de70df-5bc8-419e-b6f4-7445ea9aade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:28.270508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de70df-5bc8-419e-b6f4-7445ea9aade7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:28.270794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de70df-5bc8-419e-b6f4-7445ea9aade7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:28.271072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de70df-5bc8-419e-b6f4-7445ea9aade7 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:28.271708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de70df-5bc8-419e-b6f4-7445ea9aade7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:28.272013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de70df-5bc8-419e-b6f4-7445ea9aade7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:28.273386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de70df-5bc8-419e-b6f4-7445ea9aade7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:28.280905 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 587/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 30 09:24:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:30.441400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d352c41-4fc7-42c0-920c-b6769c5d867d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:30.441895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d352c41-4fc7-42c0-920c-b6769c5d867d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:30.442009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d352c41-4fc7-42c0-920c-b6769c5d867d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:30.443310 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 580/1167] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:30.448481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576889e-7ca4-4777-b640-5709ef753f29 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:30.448874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576889e-7ca4-4777-b640-5709ef753f29 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:30.449044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576889e-7ca4-4777-b640-5709ef753f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:30.466788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1576889e-7ca4-4777-b640-5709ef753f29 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:30.468540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1576889e-7ca4-4777-b640-5709ef753f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:30.533720 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 588/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:30.803872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d352c41-4fc7-42c0-920c-b6769c5d867d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:30.852855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:30.902537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d352c41-4fc7-42c0-920c-b6769c5d867d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:30.935095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ce515-0e32-45fc-8142-8fc7e5125002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:30.935689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce515-0e32-45fc-8142-8fc7e5125002 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:30.935968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce515-0e32-45fc-8142-8fc7e5125002 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:30.936319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce515-0e32-45fc-8142-8fc7e5125002 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:30.937120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ce515-0e32-45fc-8142-8fc7e5125002 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:30.938513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ce515-0e32-45fc-8142-8fc7e5125002 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:30.954704 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 581/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Fri Sep 30 09:24:30 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:24:33.113476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae967c5-3ac2-446c-8f01-d5bc27bf2857 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:33.114059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae967c5-3ac2-446c-8f01-d5bc27bf2857 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:33.114059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae967c5-3ac2-446c-8f01-d5bc27bf2857 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:33.115392 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 589/1170] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:33.120521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4d5bb-d184-4c09-ae8d-97a061a62ad3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:33.120926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4d5bb-d184-4c09-ae8d-97a061a62ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:33.121073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4d5bb-d184-4c09-ae8d-97a061a62ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:33.139419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e0c4d5bb-d184-4c09-ae8d-97a061a62ad3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:33.140503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e0c4d5bb-d184-4c09-ae8d-97a061a62ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:33.204460 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 582/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:24:33.477050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae967c5-3ac2-446c-8f01-d5bc27bf2857 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:33.501777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:33.576360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae967c5-3ac2-446c-8f01-d5bc27bf2857 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:33.611005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e1d80-0be2-4d3c-8534-60d3860b5bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:33.611620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e1d80-0be2-4d3c-8534-60d3860b5bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:33.611898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e1d80-0be2-4d3c-8534-60d3860b5bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:33.612168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e1d80-0be2-4d3c-8534-60d3860b5bd8 None admin] PATH_INFO: `/v3/domains/0ba54cacea534410a860680f49f85374` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:33.622119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e1d80-0be2-4d3c-8534-60d3860b5bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ba54cacea534410a860680f49f85374)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:33.623590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e1d80-0be2-4d3c-8534-60d3860b5bd8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:33.625163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 590/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Fri Sep 30 09:24:33 2022] GET /identity/v3/domains/0ba54cacea534410a860680f49f85374 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:33.630940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4d5bb-d184-4c09-ae8d-97a061a62ad3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:33.669620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:33.727358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4d5bb-d184-4c09-ae8d-97a061a62ad3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:33.758694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a2f36-fc4b-4d53-b4bd-90c42ae87845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:33.759291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a2f36-fc4b-4d53-b4bd-90c42ae87845 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:33.759565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a2f36-fc4b-4d53-b4bd-90c42ae87845 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:33.759832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a2f36-fc4b-4d53-b4bd-90c42ae87845 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:33.760478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a2f36-fc4b-4d53-b4bd-90c42ae87845 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:33.761914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a2f36-fc4b-4d53-b4bd-90c42ae87845 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:33.788125 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 583/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:24:33 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:24:36.011175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bebe39-2f8d-488e-b60a-451ce34a0bad None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:36.011752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bebe39-2f8d-488e-b60a-451ce34a0bad None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:36.011752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bebe39-2f8d-488e-b60a-451ce34a0bad None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:36.013077 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 591/1174] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:36.019084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5c83-a271-4638-83e7-74eaccc64055 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:36.019461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5c83-a271-4638-83e7-74eaccc64055 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:36.019641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5c83-a271-4638-83e7-74eaccc64055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:36.038494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-59da5c83-a271-4638-83e7-74eaccc64055 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:36.039671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-59da5c83-a271-4638-83e7-74eaccc64055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:36.104039 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 584/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:36.381686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bebe39-2f8d-488e-b60a-451ce34a0bad None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:36.463302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:36.482679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bebe39-2f8d-488e-b60a-451ce34a0bad None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:36.517971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ea030-8018-4bd5-bd7b-023c141c591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:36.518604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ea030-8018-4bd5-bd7b-023c141c591d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:36.518883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ea030-8018-4bd5-bd7b-023c141c591d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:36.519153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ea030-8018-4bd5-bd7b-023c141c591d None admin] PATH_INFO: `/v3/domains/0ba54cacea534410a860680f49f85374` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:36.520255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ea030-8018-4bd5-bd7b-023c141c591d None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ba54cacea534410a860680f49f85374)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:36.521722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ea030-8018-4bd5-bd7b-023c141c591d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:36.525671 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 592/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Fri Sep 30 09:24:36 2022] GET /identity/v3/domains/0ba54cacea534410a860680f49f85374 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:36.529381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da5c83-a271-4638-83e7-74eaccc64055 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:36.551293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:36.629195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da5c83-a271-4638-83e7-74eaccc64055 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:36.662969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea21f2-64eb-4d91-94c2-30798ea1a648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:36.663497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea21f2-64eb-4d91-94c2-30798ea1a648 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:36.663776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea21f2-64eb-4d91-94c2-30798ea1a648 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:36.664049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea21f2-64eb-4d91-94c2-30798ea1a648 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:36.664706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea21f2-64eb-4d91-94c2-30798ea1a648 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:36.666167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea21f2-64eb-4d91-94c2-30798ea1a648 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:36.689172 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 585/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:24:36 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:24:38.914194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2cced-c006-4ea0-9948-0f0e6c0e254b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:38.914726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2cced-c006-4ea0-9948-0f0e6c0e254b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:38.914726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2cced-c006-4ea0-9948-0f0e6c0e254b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:38.916077 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 593/1178] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:38.921264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f036b3-00d8-4c96-928d-7a255cd80a89 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:38.921702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f036b3-00d8-4c96-928d-7a255cd80a89 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:38.921838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f036b3-00d8-4c96-928d-7a255cd80a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:38.940664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d6f036b3-00d8-4c96-928d-7a255cd80a89 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:38.941899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d6f036b3-00d8-4c96-928d-7a255cd80a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:39.005797 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 586/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:39.278352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2cced-c006-4ea0-9948-0f0e6c0e254b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:39.377187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2cced-c006-4ea0-9948-0f0e6c0e254b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:39.410963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f2c45-a3a9-4e76-afe3-0f562340fb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:39.411588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f2c45-a3a9-4e76-afe3-0f562340fb58 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:39.411880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f2c45-a3a9-4e76-afe3-0f562340fb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:39.412148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f2c45-a3a9-4e76-afe3-0f562340fb58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:39.422308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f2c45-a3a9-4e76-afe3-0f562340fb58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6f1f2c45-a3a9-4e76-afe3-0f562340fb58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:24:39.427075 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:24:39.428042 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:24:39.428881 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 594/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:24:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:24:39.434201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f036b3-00d8-4c96-928d-7a255cd80a89 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:39.442678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:39.531144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f036b3-00d8-4c96-928d-7a255cd80a89 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:39.563547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:39.564144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:39.564422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:39.564694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:39.565351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:39.565673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:39.567030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:39.574760 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 587/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:24:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:39.580909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f2c45-a3a9-4e76-afe3-0f562340fb58 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:39.593886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:39.680545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f2c45-a3a9-4e76-afe3-0f562340fb58 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:39.713570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d83fd3-56d2-47f6-8f5a-1393495273ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:39.714215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d83fd3-56d2-47f6-8f5a-1393495273ba None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:39.714498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d83fd3-56d2-47f6-8f5a-1393495273ba None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:39.714772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d83fd3-56d2-47f6-8f5a-1393495273ba None admin] PATH_INFO: `/v3/users/f6342e1f1bda4cc28636c2f13e5880f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:39.727211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d83fd3-56d2-47f6-8f5a-1393495273ba None admin] RBAC: Authorizing `identity:get_user(user_id=f6342e1f1bda4cc28636c2f13e5880f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:39.728683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d83fd3-56d2-47f6-8f5a-1393495273ba None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:39.730346 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 595/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:39 2022] GET /identity/v3/users/f6342e1f1bda4cc28636c2f13e5880f2 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:39.736574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:39.801249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:39.833591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdc657-6c66-424a-9767-b3f9c181ed53 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:39.865755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19dbc5-27b5-4c7f-981e-24f4942505bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:39.866349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19dbc5-27b5-4c7f-981e-24f4942505bf None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:39.866630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19dbc5-27b5-4c7f-981e-24f4942505bf None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:39.866901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19dbc5-27b5-4c7f-981e-24f4942505bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:39.931051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19dbc5-27b5-4c7f-981e-24f4942505bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:39.932497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19dbc5-27b5-4c7f-981e-24f4942505bf None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:39.999173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:40.033197 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 588/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:24:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:24:40.039097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d83fd3-56d2-47f6-8f5a-1393495273ba None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:40.103323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:40.135879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d83fd3-56d2-47f6-8f5a-1393495273ba None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:40.168839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4da35-411a-4e0b-b6c3-93c3f6a07f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:40.169458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4da35-411a-4e0b-b6c3-93c3f6a07f18 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:40.169765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4da35-411a-4e0b-b6c3-93c3f6a07f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:40.170041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4da35-411a-4e0b-b6c3-93c3f6a07f18 None admin] PATH_INFO: `/v3/projects/9c0fea27ec7d4e59920051a0b0ec4873` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:40.171675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4da35-411a-4e0b-b6c3-93c3f6a07f18 None admin] RBAC: Authorizing `identity:get_project(project_id=9c0fea27ec7d4e59920051a0b0ec4873)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:40.173128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4da35-411a-4e0b-b6c3-93c3f6a07f18 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:40.174822 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 596/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:40 2022] GET /identity/v3/projects/9c0fea27ec7d4e59920051a0b0ec4873 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:40.181027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19dbc5-27b5-4c7f-981e-24f4942505bf None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:40.278783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19dbc5-27b5-4c7f-981e-24f4942505bf None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:40.311841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d907a6f-8cc0-4e04-9f1a-27693d82b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:40.312353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d907a6f-8cc0-4e04-9f1a-27693d82b48c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:40.312626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d907a6f-8cc0-4e04-9f1a-27693d82b48c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:40.312889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d907a6f-8cc0-4e04-9f1a-27693d82b48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:40.313593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d907a6f-8cc0-4e04-9f1a-27693d82b48c None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=9c0fea27ec7d4e59920051a0b0ec4873, user.id=f6342e1f1bda4cc28636c2f13e5880f2) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:40.313927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d907a6f-8cc0-4e04-9f1a-27693d82b48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:40.315327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d907a6f-8cc0-4e04-9f1a-27693d82b48c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:40.320420 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 589/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:24:40 2022] GET /identity/v3/role_assignments?user.id=f6342e1f1bda4cc28636c2f13e5880f2&scope.project.id=9c0fea27ec7d4e59920051a0b0ec4873&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:42.475726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a6d6d-d1cf-4315-8007-01d07cfd575c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:42.476222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a6d6d-d1cf-4315-8007-01d07cfd575c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:42.476325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a6d6d-d1cf-4315-8007-01d07cfd575c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:42.477587 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 597/1186] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:42.482846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1db82-aa5f-41ec-824e-d8416fb8bcc7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:42.483226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1db82-aa5f-41ec-824e-d8416fb8bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:42.483401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1db82-aa5f-41ec-824e-d8416fb8bcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:42.501658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-13f1db82-aa5f-41ec-824e-d8416fb8bcc7 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:42.502862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-13f1db82-aa5f-41ec-824e-d8416fb8bcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:42.528690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:42.568405 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 590/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:24:42.845496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a6d6d-d1cf-4315-8007-01d07cfd575c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:42.943267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a6d6d-d1cf-4315-8007-01d07cfd575c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:42.977638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2782673-1280-4033-b8c7-53d7521df046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:42.978263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782673-1280-4033-b8c7-53d7521df046 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:42.978543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782673-1280-4033-b8c7-53d7521df046 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:42.978815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782673-1280-4033-b8c7-53d7521df046 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:42.981350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:42.989839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2782673-1280-4033-b8c7-53d7521df046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d2782673-1280-4033-b8c7-53d7521df046 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:24:42.994559 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:24:42.995645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:24:42.996689 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 598/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:24:43.001928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1db82-aa5f-41ec-824e-d8416fb8bcc7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:43.099815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1db82-aa5f-41ec-824e-d8416fb8bcc7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:43.131730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:43.132318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:43.132594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:43.132864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:43.133513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:43.133838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:43.135181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:43.143537 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 591/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:24:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:43.149950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2782673-1280-4033-b8c7-53d7521df046 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:43.245412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2782673-1280-4033-b8c7-53d7521df046 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:43.278609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999447e0-d16c-42f0-9e14-09b34b5d7741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:43.279223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999447e0-d16c-42f0-9e14-09b34b5d7741 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:43.279532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999447e0-d16c-42f0-9e14-09b34b5d7741 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:43.279802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999447e0-d16c-42f0-9e14-09b34b5d7741 None admin] PATH_INFO: `/v3/users/f6342e1f1bda4cc28636c2f13e5880f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:43.280996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999447e0-d16c-42f0-9e14-09b34b5d7741 None admin] RBAC: Authorizing `identity:get_user(user_id=f6342e1f1bda4cc28636c2f13e5880f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:43.282453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999447e0-d16c-42f0-9e14-09b34b5d7741 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:43.284016 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 599/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:43 2022] GET /identity/v3/users/f6342e1f1bda4cc28636c2f13e5880f2 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:43.289831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:43.314789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:43.387043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbe2c5-2040-4fb2-94db-d9ba21a77d72 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:43.419703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410548c-4566-4b88-884b-bbe18bf7d629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:43.420467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410548c-4566-4b88-884b-bbe18bf7d629 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:43.420780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410548c-4566-4b88-884b-bbe18bf7d629 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:43.421053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410548c-4566-4b88-884b-bbe18bf7d629 None admin] PATH_INFO: `/v3/projects/9c0fea27ec7d4e59920051a0b0ec4873` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:43.422188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410548c-4566-4b88-884b-bbe18bf7d629 None admin] RBAC: Authorizing `identity:get_project(project_id=9c0fea27ec7d4e59920051a0b0ec4873)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:43.423609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410548c-4566-4b88-884b-bbe18bf7d629 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:43.425218 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 592/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:43 2022] GET /identity/v3/projects/9c0fea27ec7d4e59920051a0b0ec4873 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:43.431122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999447e0-d16c-42f0-9e14-09b34b5d7741 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:43.491521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:43.529035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999447e0-d16c-42f0-9e14-09b34b5d7741 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:43.562041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2b193-02af-4e65-9837-80b1df6c451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:43.562678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2b193-02af-4e65-9837-80b1df6c451e None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:43.562953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2b193-02af-4e65-9837-80b1df6c451e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:43.563221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2b193-02af-4e65-9837-80b1df6c451e None admin] PATH_INFO: `/v3/projects/9c0fea27ec7d4e59920051a0b0ec4873/users/f6342e1f1bda4cc28636c2f13e5880f2/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:43.565098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2b193-02af-4e65-9837-80b1df6c451e None admin] RBAC: Authorizing `identity:create_grant(project_id=9c0fea27ec7d4e59920051a0b0ec4873, user_id=f6342e1f1bda4cc28636c2f13e5880f2, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:43.566564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2b193-02af-4e65-9837-80b1df6c451e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:43.578822 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 600/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:24:43 2022] PUT /identity/v3/projects/9c0fea27ec7d4e59920051a0b0ec4873/users/f6342e1f1bda4cc28636c2f13e5880f2/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:24:45.700704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e732f44-fd87-4d5a-b177-1e35dc42d441 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:45.701272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e732f44-fd87-4d5a-b177-1e35dc42d441 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:45.701272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e732f44-fd87-4d5a-b177-1e35dc42d441 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:45.702663 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 593/1193] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:24:45.707853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecc504-3c92-45c2-b270-435ef25f2cf2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:45.708225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecc504-3c92-45c2-b270-435ef25f2cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:45.708445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecc504-3c92-45c2-b270-435ef25f2cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:45.726259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-71ecc504-3c92-45c2-b270-435ef25f2cf2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:24:45.727448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-71ecc504-3c92-45c2-b270-435ef25f2cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:24:45.791845 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 601/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:24:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:24:46.064230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e732f44-fd87-4d5a-b177-1e35dc42d441 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:46.152576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:46.162165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e732f44-fd87-4d5a-b177-1e35dc42d441 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:46.194501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0351e4-0bc2-4315-8c47-933b7fc2aba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:46.195118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0351e4-0bc2-4315-8c47-933b7fc2aba7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:46.195398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0351e4-0bc2-4315-8c47-933b7fc2aba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:46.195671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0351e4-0bc2-4315-8c47-933b7fc2aba7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:46.205939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0351e4-0bc2-4315-8c47-933b7fc2aba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-bf0351e4-0bc2-4315-8c47-933b7fc2aba7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:24:46.210677 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:24:46.211768 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:24:46.212652 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 594/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:24:46.217972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecc504-3c92-45c2-b270-435ef25f2cf2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:46.272024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:46.316508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecc504-3c92-45c2-b270-435ef25f2cf2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:46.349362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:46.349986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:46.350269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:46.350543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:46.351201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:46.351507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:46.352880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:46.361311 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 602/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:24:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:24:46.367300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0351e4-0bc2-4315-8c47-933b7fc2aba7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:46.464360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0351e4-0bc2-4315-8c47-933b7fc2aba7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:46.496892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e9bd3-0005-49b2-894a-7cbc2dd50fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:46.497508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9bd3-0005-49b2-894a-7cbc2dd50fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:46.497817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9bd3-0005-49b2-894a-7cbc2dd50fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:46.498639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e9bd3-0005-49b2-894a-7cbc2dd50fc6 None admin] PATH_INFO: `/v3/users/f6342e1f1bda4cc28636c2f13e5880f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:46.499745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e9bd3-0005-49b2-894a-7cbc2dd50fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=f6342e1f1bda4cc28636c2f13e5880f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:46.501161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e9bd3-0005-49b2-894a-7cbc2dd50fc6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:46.502744 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 595/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 30 09:24:46 2022] GET /identity/v3/users/f6342e1f1bda4cc28636c2f13e5880f2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:46.508566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:46.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deefe6e-261d-48e4-980d-6dec3157b178 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:46.622809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:46.641040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e340b-2e31-44db-bec3-3ec34ee63c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:46.641660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e340b-2e31-44db-bec3-3ec34ee63c03 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:46.641951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e340b-2e31-44db-bec3-3ec34ee63c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:46.642226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e340b-2e31-44db-bec3-3ec34ee63c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:46.710446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e340b-2e31-44db-bec3-3ec34ee63c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:46.711905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e340b-2e31-44db-bec3-3ec34ee63c03 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:46.807037 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 603/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:24:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:24:46.813233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e9bd3-0005-49b2-894a-7cbc2dd50fc6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:46.834440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:46.909374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e9bd3-0005-49b2-894a-7cbc2dd50fc6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:46.942163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0448d72-5d05-4813-9998-5640050dded2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:46.942781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0448d72-5d05-4813-9998-5640050dded2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:46.943055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0448d72-5d05-4813-9998-5640050dded2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:46.943324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0448d72-5d05-4813-9998-5640050dded2 None admin] PATH_INFO: `/v3/projects/9c0fea27ec7d4e59920051a0b0ec4873` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:46.944407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0448d72-5d05-4813-9998-5640050dded2 None admin] RBAC: Authorizing `identity:get_project(project_id=9c0fea27ec7d4e59920051a0b0ec4873)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:46.945858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0448d72-5d05-4813-9998-5640050dded2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:46.947448 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 596/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 30 09:24:46 2022] GET /identity/v3/projects/9c0fea27ec7d4e59920051a0b0ec4873 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:46.953760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e340b-2e31-44db-bec3-3ec34ee63c03 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:24:46.971063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:47.052428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e340b-2e31-44db-bec3-3ec34ee63c03 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:24:47.084315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea31c5-b5e5-4974-b17e-f5759cf1563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:24:47.084905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea31c5-b5e5-4974-b17e-f5759cf1563f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:24:47.085180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea31c5-b5e5-4974-b17e-f5759cf1563f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:24:47.085447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea31c5-b5e5-4974-b17e-f5759cf1563f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:24:47.086190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea31c5-b5e5-4974-b17e-f5759cf1563f None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=9c0fea27ec7d4e59920051a0b0ec4873, user.id=f6342e1f1bda4cc28636c2f13e5880f2) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:24:47.086493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea31c5-b5e5-4974-b17e-f5759cf1563f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:24:47.087918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea31c5-b5e5-4974-b17e-f5759cf1563f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:24:47.094114 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 604/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:24:46 2022] GET /identity/v3/role_assignments?user.id=f6342e1f1bda4cc28636c2f13e5880f2&scope.project.id=9c0fea27ec7d4e59920051a0b0ec4873&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:24:56.938058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=56 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:24:57.090949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=60 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:43.317145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa1021-ee5c-4de0-9fd3-868ea3b9b13a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:43.317774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa1021-ee5c-4de0-9fd3-868ea3b9b13a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:43.318010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa1021-ee5c-4de0-9fd3-868ea3b9b13a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:43.319561 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 597/1201] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:25:43.325375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706762e-5bf7-497f-969a-e2d2f7a052cb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:43.325922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706762e-5bf7-497f-969a-e2d2f7a052cb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:43.326003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706762e-5bf7-497f-969a-e2d2f7a052cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:43.351735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0706762e-5bf7-497f-969a-e2d2f7a052cb None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:25:43.352927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0706762e-5bf7-497f-969a-e2d2f7a052cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:25:43.457672 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 605/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:25:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:25:43.736010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa1021-ee5c-4de0-9fd3-868ea3b9b13a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:43.866427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa1021-ee5c-4de0-9fd3-868ea3b9b13a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:43.908457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:43.909252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:43.909537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:43.909858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:43.921232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:43.922906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:25:43.945925 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:43.947067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:25:43.948045 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:25:43.948045 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:43.948045 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:25:43.948045 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 598/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:43.953998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706762e-5bf7-497f-969a-e2d2f7a052cb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:44.064873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706762e-5bf7-497f-969a-e2d2f7a052cb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:44.097764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:44.104105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70332aeb-80c9-40f1-9082-91a115983d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:44.104950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70332aeb-80c9-40f1-9082-91a115983d14 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:44.105519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70332aeb-80c9-40f1-9082-91a115983d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:44.105519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70332aeb-80c9-40f1-9082-91a115983d14 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:44.106184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70332aeb-80c9-40f1-9082-91a115983d14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:44.106493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70332aeb-80c9-40f1-9082-91a115983d14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:44.107887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70332aeb-80c9-40f1-9082-91a115983d14 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:44.130490 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 606/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:25:44.138696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:44.206030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:44.262841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56442e-47dc-43d5-bfbb-90de25e25c31 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:44.302127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240ecf0-47ee-43be-a600-65e9c81c4824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:44.302758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240ecf0-47ee-43be-a600-65e9c81c4824 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:44.303067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240ecf0-47ee-43be-a600-65e9c81c4824 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:44.303315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240ecf0-47ee-43be-a600-65e9c81c4824 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:44.303975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240ecf0-47ee-43be-a600-65e9c81c4824 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:44.305406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240ecf0-47ee-43be-a600-65e9c81c4824 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:44.345770 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 599/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 30 09:25:44 2022] POST /identity/v3/users => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:25:46.677205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946fdeb-d26c-48c2-b955-2eb3b737aaa4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:46.677785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946fdeb-d26c-48c2-b955-2eb3b737aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:46.677785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946fdeb-d26c-48c2-b955-2eb3b737aaa4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:46.680221 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 607/1206] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:25:46.685302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcc044-46e1-4304-aa39-705bf7fc31e1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:46.685788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcc044-46e1-4304-aa39-705bf7fc31e1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:46.685882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcc044-46e1-4304-aa39-705bf7fc31e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:46.704612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b5fcc044-46e1-4304-aa39-705bf7fc31e1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:25:46.706053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b5fcc044-46e1-4304-aa39-705bf7fc31e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:25:46.785674 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 600/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:25:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:25:47.058407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946fdeb-d26c-48c2-b955-2eb3b737aaa4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:47.172955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946fdeb-d26c-48c2-b955-2eb3b737aaa4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:47.212705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb039d3-9419-4cb0-b2bc-2183896f00d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:47.213354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb039d3-9419-4cb0-b2bc-2183896f00d3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:47.213675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb039d3-9419-4cb0-b2bc-2183896f00d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:47.214916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb039d3-9419-4cb0-b2bc-2183896f00d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:47.226021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb039d3-9419-4cb0-b2bc-2183896f00d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4bb039d3-9419-4cb0-b2bc-2183896f00d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:25:47.231005 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:47.233816 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:47.234682 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 608/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:25:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:47.238708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcc044-46e1-4304-aa39-705bf7fc31e1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:47.286123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:47.361803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcc044-46e1-4304-aa39-705bf7fc31e1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:47.402061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:47.405466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:47.405884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:47.406307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:47.407033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:47.407403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:47.409151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:47.425011 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 601/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:25:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:47.426790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb039d3-9419-4cb0-b2bc-2183896f00d3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:47.490380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:47.554305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb039d3-9419-4cb0-b2bc-2183896f00d3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:47.596267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:47.597083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:47.597479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:47.598682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:47.611091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:25:47.628531 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:47.634485 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:25:47.635453 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:25:47.635453 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:47.635453 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:47.635453 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 609/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:47.635745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:47.750463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:47.754183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe67f2-a6f3-4a44-9e9e-de9d12eff90d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:47.795217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:47.795826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:47.796105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:47.796375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:47.797346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:47.797671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:47.799050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:47.813393 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 602/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:47.819012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:47.948192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68dbcd-b5e2-422b-8337-1d479556f30c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:47.950687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:47.993662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:47.994516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:47.994805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:47.995235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] PATH_INFO: `/v3/users/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:48.004345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:48.006237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:25:48.020243 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:48.022035 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:48.023060 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 610/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:48.028519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:48.163629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107afe8-745a-4ad4-9c8a-18887b8bed5b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:48.202164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:48.202742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:48.203076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:48.203404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:48.204434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:48.204855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:48.206766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:48.236001 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 603/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 30 09:25:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:48.237251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:48.358088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1e88a-bb12-45bd-b2f1-121df3f1d9db None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:48.402910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642d2c-c2b6-410f-b43c-7b626b93fce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:48.403535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642d2c-c2b6-410f-b43c-7b626b93fce5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:48.403817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642d2c-c2b6-410f-b43c-7b626b93fce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:48.404090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14642d2c-c2b6-410f-b43c-7b626b93fce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:48.438227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:48.483510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14642d2c-c2b6-410f-b43c-7b626b93fce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:48.485219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14642d2c-c2b6-410f-b43c-7b626b93fce5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:48.611470 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 611/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:25:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:25:48.613001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:48.636741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:48.739953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba77da-7150-4f97-998f-b9242462705c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:48.793195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:48.793890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:48.794143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:48.794414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:48.805863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:48.807449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:25:48.826009 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:48.827396 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:25:48.828305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:25:48.828305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:48.828305 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:25:48.828305 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 604/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:48.833371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642d2c-c2b6-410f-b43c-7b626b93fce5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:48.904551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:48.951055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14642d2c-c2b6-410f-b43c-7b626b93fce5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:48.999055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:49.000066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:49.000613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:49.000938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:49.001942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:49.002292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:49.004168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:49.020988 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 612/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:49.027838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:49.116688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:49.157039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e808ca-9269-4cec-8504-e26f218ed5cb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:49.195633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:49.196357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:49.196698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:49.196934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:49.205893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:49.207479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:25:49.221098 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:49.222264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:49.223700 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:25:49.223700 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:25:49.223700 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:25:49.223700 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:25:49.223700 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:25:49.223700 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:25:49.223700 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:25:49.223700 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:25:49.229189 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 605/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:49.229534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:49.362289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29972ac3-72c0-494f-8238-71de8462b9a2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:49.390472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:49.405109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef161b87-1040-45a5-828f-f9e21614d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:49.405773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef161b87-1040-45a5-828f-f9e21614d29d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:49.406048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef161b87-1040-45a5-828f-f9e21614d29d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:49.406316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef161b87-1040-45a5-828f-f9e21614d29d None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:49.407228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef161b87-1040-45a5-828f-f9e21614d29d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:49.407332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef161b87-1040-45a5-828f-f9e21614d29d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:49.408786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef161b87-1040-45a5-828f-f9e21614d29d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:49.425105 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 613/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:49.431062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:49.545084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e218f-c37d-4d07-bf2d-5d4804dedf7e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:49.574819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:49.583110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bae0ee-7ffc-4e9f-922b-6ba97c6e84a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:49.583696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae0ee-7ffc-4e9f-922b-6ba97c6e84a7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:49.583997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae0ee-7ffc-4e9f-922b-6ba97c6e84a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:49.584241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae0ee-7ffc-4e9f-922b-6ba97c6e84a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:49.584952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bae0ee-7ffc-4e9f-922b-6ba97c6e84a7 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=5989c8a1e28a43d3bc16970850ce49f4) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:49.585252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bae0ee-7ffc-4e9f-922b-6ba97c6e84a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:49.586727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bae0ee-7ffc-4e9f-922b-6ba97c6e84a7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:49.592238 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 606/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:25:49 2022] GET /identity/v3/role_assignments?user.id=5989c8a1e28a43d3bc16970850ce49f4&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:51.753122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59e419-c657-4ec5-b18c-dd326bfaaeb7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:51.754202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59e419-c657-4ec5-b18c-dd326bfaaeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:51.754202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59e419-c657-4ec5-b18c-dd326bfaaeb7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:51.757810 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 614/1220] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:25:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:25:51.764457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9e07f-c36b-41cc-8092-87dbdd933747 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:51.764929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9e07f-c36b-41cc-8092-87dbdd933747 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:51.765029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9e07f-c36b-41cc-8092-87dbdd933747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:51.785979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4cf9e07f-c36b-41cc-8092-87dbdd933747 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:25:51.787484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4cf9e07f-c36b-41cc-8092-87dbdd933747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:25:51.862078 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 607/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:25:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:25:52.165246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59e419-c657-4ec5-b18c-dd326bfaaeb7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:52.274851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59e419-c657-4ec5-b18c-dd326bfaaeb7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:52.311663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a131b2-4570-467f-88aa-7d12a34216d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:52.312331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a131b2-4570-467f-88aa-7d12a34216d1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:52.312615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a131b2-4570-467f-88aa-7d12a34216d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:52.312889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a131b2-4570-467f-88aa-7d12a34216d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:52.323568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a131b2-4570-467f-88aa-7d12a34216d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d6a131b2-4570-467f-88aa-7d12a34216d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:25:52.328602 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:52.329688 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:52.330583 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 615/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:25:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:52.336224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9e07f-c36b-41cc-8092-87dbdd933747 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:52.446942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9e07f-c36b-41cc-8092-87dbdd933747 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:52.472026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:52.483602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:52.484216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:52.484497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:52.484768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:52.485433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:52.485903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:52.487272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:52.495940 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 608/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:25:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:52.501501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a131b2-4570-467f-88aa-7d12a34216d1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:52.547418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:52.618495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a131b2-4570-467f-88aa-7d12a34216d1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:52.659373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:52.660061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:52.660390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:52.660692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:52.675603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:52.677222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:25:52.696524 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:52.698093 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:25:52.698989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:25:52.698989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:52.698989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:52.698989 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 616/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:52.704250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:52.820567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b291d-3914-4703-a268-a83e284cbfa8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:52.859786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:52.860418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:52.860703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:52.860997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:52.861667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:52.862052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:52.863571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:52.876285 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 609/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:52.882240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:52.982904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:52.991057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f760791-6a06-454e-9ce0-08a35fdff115 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:53.030751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:53.031797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:53.032186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:53.032609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:53.043297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:53.044950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:25:53.059401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:53.060693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:53.062034 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 617/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:53.070964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:53.124523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:53.182029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356d4ef-45ac-473a-91fd-32390f9ba1eb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:53.227156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:53.227952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:53.228272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:53.228575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:53.230152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:53.230472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:53.231927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:53.254185 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 610/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 30 09:25:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:25:53.260322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:53.372727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1380b-d482-4d99-82ea-53c561dd29bd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:53.412046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:53.412673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:53.412957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:53.413229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:53.424550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:53.426517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:25:53.441253 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:53.444118 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:25:53.445347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:25:53.445347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:53.445347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:53.445347 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 618/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:53.448721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:53.550792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27e7bb-728b-4a41-9210-4d11621430e9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:53.553384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:53.588074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:53.589270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:53.589550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:53.589931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:53.590598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:53.590947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:53.592407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:53.604888 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 611/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:53.610668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:53.620749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:53.721181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c7e7b-a239-4172-a87e-5e5ef6c66101 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:53.759011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:53.759665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:53.759925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:53.760195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:53.767502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:53.769017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:25:53.781424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:53.782739 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:53.784009 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 619/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:25:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:25:53.788954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:53.900792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258928cc-5c98-4d6c-8067-ae2e6aaec15f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:53.937805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c5748-9cda-4665-b2bb-64703186cc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:53.938409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c5748-9cda-4665-b2bb-64703186cc62 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:53.938684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c5748-9cda-4665-b2bb-64703186cc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:53.938951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c5748-9cda-4665-b2bb-64703186cc62 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:53.939631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c5748-9cda-4665-b2bb-64703186cc62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:53.939930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c5748-9cda-4665-b2bb-64703186cc62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:53.941335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c5748-9cda-4665-b2bb-64703186cc62 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:53.954923 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 612/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:25:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:53.961076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:54.033241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:54.074704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d8404-e4ed-4d0e-afbe-234426e8b5fe None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:54.109941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd98e2-57af-4f20-affd-bb477fb50fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:54.110596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd98e2-57af-4f20-affd-bb477fb50fcb None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:54.110874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd98e2-57af-4f20-affd-bb477fb50fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:54.111143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd98e2-57af-4f20-affd-bb477fb50fcb None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/5989c8a1e28a43d3bc16970850ce49f4/roles/170f0bd5d6914cb8ad93877cd5cd07bb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:54.126629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd98e2-57af-4f20-affd-bb477fb50fcb None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=5989c8a1e28a43d3bc16970850ce49f4, role_id=170f0bd5d6914cb8ad93877cd5cd07bb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:54.128198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd98e2-57af-4f20-affd-bb477fb50fcb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:54.140686 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 620/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:25:53 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/5989c8a1e28a43d3bc16970850ce49f4/roles/170f0bd5d6914cb8ad93877cd5cd07bb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:25:56.452013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ec580-bee8-4f32-a347-78b293f9ee01 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:56.452570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ec580-bee8-4f32-a347-78b293f9ee01 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:56.452570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ec580-bee8-4f32-a347-78b293f9ee01 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:56.453944 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 613/1233] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:25:56.460365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db673-7063-490e-8bdc-d1f7f1a8496c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:56.460798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db673-7063-490e-8bdc-d1f7f1a8496c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:56.460938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db673-7063-490e-8bdc-d1f7f1a8496c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:56.482281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9d9db673-7063-490e-8bdc-d1f7f1a8496c None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:25:56.483753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9d9db673-7063-490e-8bdc-d1f7f1a8496c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:25:56.563169 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 621/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:25:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:25:56.876529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ec580-bee8-4f32-a347-78b293f9ee01 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:56.904536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:56.991083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ec580-bee8-4f32-a347-78b293f9ee01 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:57.033888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd33adf-52c3-4a35-8362-f08318f98ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:57.034605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd33adf-52c3-4a35-8362-f08318f98ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:57.034948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd33adf-52c3-4a35-8362-f08318f98ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:57.035173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd33adf-52c3-4a35-8362-f08318f98ed2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:57.049011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd33adf-52c3-4a35-8362-f08318f98ed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-dbd33adf-52c3-4a35-8362-f08318f98ed2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:25:57.056904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:57.059293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:25:57.060168 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 614/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 30 09:25:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:57.067339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db673-7063-490e-8bdc-d1f7f1a8496c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:57.119509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:57.180958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db673-7063-490e-8bdc-d1f7f1a8496c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:57.217386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:57.218025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:57.218307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:57.218579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:57.219318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:57.219569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:57.221160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:57.232368 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 622/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 30 09:25:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:57.238610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd33adf-52c3-4a35-8362-f08318f98ed2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:57.330578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:57.347205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd33adf-52c3-4a35-8362-f08318f98ed2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:57.383908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:57.384606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:57.384972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:57.385331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:57.397409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:57.399189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:25:57.417830 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:57.418864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:25:57.419764 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:25:57.419764 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:57.419764 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:25:57.419764 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 615/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:57.425186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:57.545239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd6892-4630-40fc-954b-ccd49f0a552c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:57.554311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:57.581264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:57.581907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:57.582193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:57.582468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:57.583110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:57.583416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:57.584802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:57.598424 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 623/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:57.604790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:57.728120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d48f8-bb64-415f-9462-5a14a5a76d38 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:57.770975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:57.771616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:57.771886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:57.772166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:57.775176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:57.781141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:57.782947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:25:57.795042 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:57.796406 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:25:57.797555 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 616/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:57.802243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:57.912200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a92af3-4f12-4e88-b0c8-50d6c18c2f7f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:57.951391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:57.952171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:57.952559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:57.952892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:57.953698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:57.954043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:57.955865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:57.974497 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 624/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 30 09:25:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:57.981222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:58.096801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959edaa-d151-4985-a1da-ca573ba52b77 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:58.131744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee033baa-1e65-4d1e-99d5-05c6516c170f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:58.132350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee033baa-1e65-4d1e-99d5-05c6516c170f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:58.132629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee033baa-1e65-4d1e-99d5-05c6516c170f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:58.132901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee033baa-1e65-4d1e-99d5-05c6516c170f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:58.201127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:58.208243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee033baa-1e65-4d1e-99d5-05c6516c170f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:58.209717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee033baa-1e65-4d1e-99d5-05c6516c170f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:58.319903 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 617/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:25:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:25:58.326270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:58.354150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:58.434447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca314cce-70cc-44f3-a5bb-e54da551bb73 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:58.479635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:58.480382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:58.480664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:58.480933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:58.494079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:58.496357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:25:58.518348 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:58.519553 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:25:58.520477 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:25:58.520477 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:25:58.520477 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:58.520477 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 625/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:58.525521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee033baa-1e65-4d1e-99d5-05c6516c170f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:58.632955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:58.639894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee033baa-1e65-4d1e-99d5-05c6516c170f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:58.678537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:58.679167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:58.679449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:58.679728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:58.680459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:58.680785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:58.682779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:58.694791 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 618/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:58.700669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:58.790458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:25:58.816647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806587-29e5-4376-bc9d-333df1cc3bd7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:58.858023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:58.858985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:58.859454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:58.859906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:58.868346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:58.870188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:25:58.882919 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:25:58.884013 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:25:58.884921 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 626/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:25:58.891086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:59.020637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b282f66-baa9-49f9-afc8-6fff2b627478 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:59.065652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f494a-84d4-4cce-a24a-310c1d57517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:59.066321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f494a-84d4-4cce-a24a-310c1d57517c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:59.066610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f494a-84d4-4cce-a24a-310c1d57517c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:59.066912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f494a-84d4-4cce-a24a-310c1d57517c None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:59.067612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f494a-84d4-4cce-a24a-310c1d57517c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:59.067920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f494a-84d4-4cce-a24a-310c1d57517c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:59.069349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f494a-84d4-4cce-a24a-310c1d57517c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:59.083542 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 619/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:25:59.091071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:25:59.206635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856de241-3ac1-42bb-9e85-e0fe785c9b55 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:25:59.241529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0bebc-b767-4210-98d0-715989ea43b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:25:59.242160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0bebc-b767-4210-98d0-715989ea43b7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:25:59.242447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0bebc-b767-4210-98d0-715989ea43b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:25:59.242696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0bebc-b767-4210-98d0-715989ea43b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:25:59.243413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0bebc-b767-4210-98d0-715989ea43b7 None admin] RBAC: Adding query filter params (role.id=170f0bd5d6914cb8ad93877cd5cd07bb, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=5989c8a1e28a43d3bc16970850ce49f4) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:25:59.243717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0bebc-b767-4210-98d0-715989ea43b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:25:59.245949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0bebc-b767-4210-98d0-715989ea43b7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:25:59.251958 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 627/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:25:59 2022] GET /identity/v3/role_assignments?user.id=5989c8a1e28a43d3bc16970850ce49f4&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=170f0bd5d6914cb8ad93877cd5cd07bb => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:01.771643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430751b-56a9-4c52-a32f-526944b2f500 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:01.772155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430751b-56a9-4c52-a32f-526944b2f500 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:01.772155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430751b-56a9-4c52-a32f-526944b2f500 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:01.773692 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 620/1247] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:26:01.779575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c9d0c-697d-43ab-8e6c-cd407fce2ac6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:01.780036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c9d0c-697d-43ab-8e6c-cd407fce2ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:01.780115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c9d0c-697d-43ab-8e6c-cd407fce2ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:01.786753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:01.800872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-008c9d0c-697d-43ab-8e6c-cd407fce2ac6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:26:01.802149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-008c9d0c-697d-43ab-8e6c-cd407fce2ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:26:01.876129 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 628/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:26:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 30 09:26:02.200774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430751b-56a9-4c52-a32f-526944b2f500 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:02.219844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:02.312742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430751b-56a9-4c52-a32f-526944b2f500 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:02.349576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03e290-530a-45aa-af10-c815cb2a2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:02.350395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03e290-530a-45aa-af10-c815cb2a2910 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:02.350727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03e290-530a-45aa-af10-c815cb2a2910 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:02.351075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03e290-530a-45aa-af10-c815cb2a2910 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:02.363485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03e290-530a-45aa-af10-c815cb2a2910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-cd03e290-530a-45aa-af10-c815cb2a2910 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:26:02.368854 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:02.370372 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:02.371209 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 621/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:26:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:02.376946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c9d0c-697d-43ab-8e6c-cd407fce2ac6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:02.491255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c9d0c-697d-43ab-8e6c-cd407fce2ac6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:02.523759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:02.530883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:02.531651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:02.532060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:02.532438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:02.533251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:02.533582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:02.535401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:02.545735 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 629/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:26:02.552429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03e290-530a-45aa-af10-c815cb2a2910 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:02.638067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:02.666843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03e290-530a-45aa-af10-c815cb2a2910 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:02.706323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:02.706953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:02.707234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:02.707499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:02.718932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:02.720392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:26:02.735980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:02.737253 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:26:02.738356 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:26:02.738356 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:02.738356 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:02.738356 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 622/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:02.743124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:02.861305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7658d98-42aa-445c-968b-54482dbd1877 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:02.896299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:02.896905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:02.897183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:02.897454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:02.898136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:02.898708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:02.900119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:02.914356 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 630/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:02.921039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:03.031223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e753129-b0e4-43de-ab8e-4577f1fe8a50 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:03.064200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:03.074288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:03.074954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:03.075267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:03.075548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:03.085500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:03.087052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:26:03.099747 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:03.100935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:03.102516 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 623/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:03.107174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:03.167286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:03.213520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a6b3d-8199-4313-bfb0-d9bdeb47f3fa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:03.248584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:03.249189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:03.249463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:03.249760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:03.250489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:03.250851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:03.252286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:03.270227 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 631/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 30 09:26:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:03.276444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:03.377405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab098bd-add8-4345-b52d-d739e122019e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:03.413187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18071af2-41c8-4876-951f-a6248f24d244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:03.413828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18071af2-41c8-4876-951f-a6248f24d244 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:03.414108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18071af2-41c8-4876-951f-a6248f24d244 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:03.414381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18071af2-41c8-4876-951f-a6248f24d244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:03.470798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:03.483441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18071af2-41c8-4876-951f-a6248f24d244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:03.484878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18071af2-41c8-4876-951f-a6248f24d244 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:03.578017 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 624/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:26:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:26:03.583848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:03.680289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25ac54-6dc6-4448-9467-0ddcb9794624 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:03.692895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:03.714881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:03.715512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:03.715794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:03.716068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:03.725963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:03.727418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:26:03.741803 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:03.742949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:26:03.744026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:26:03.744026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:03.744026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:03.744026 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 632/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:03.748697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18071af2-41c8-4876-951f-a6248f24d244 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:03.831766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:03.843868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18071af2-41c8-4876-951f-a6248f24d244 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:03.876769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:03.877354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:03.877639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:03.877923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:03.878550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:03.878846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:03.880183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:03.890933 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 625/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:03.896415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:03.992740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fd118-c56a-4793-80c7-be6e5ec23efe None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:04.028935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:04.029769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:04.030075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:04.030422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:04.038246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:04.039726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:26:04.051834 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:04.052854 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:04.053772 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 633/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:26:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:04.059074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:04.158126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8555c-89a1-4902-863c-74b1e629db07 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:04.190733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b16d4-5863-4f7e-a200-829b8c706c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:04.191328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b16d4-5863-4f7e-a200-829b8c706c89 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:04.191602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b16d4-5863-4f7e-a200-829b8c706c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:04.191870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b16d4-5863-4f7e-a200-829b8c706c89 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:04.192555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b16d4-5863-4f7e-a200-829b8c706c89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:04.192858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b16d4-5863-4f7e-a200-829b8c706c89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:04.194305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b16d4-5863-4f7e-a200-829b8c706c89 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:04.205968 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 626/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:26:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:04.212058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:04.237770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:04.310844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765b7b9-8755-474d-9d49-a15cbbdba377 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:04.343106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b1dd3-cae0-43b1-af6b-e9a80e000f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:04.343694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1dd3-cae0-43b1-af6b-e9a80e000f43 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:04.343974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1dd3-cae0-43b1-af6b-e9a80e000f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:04.344247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b1dd3-cae0-43b1-af6b-e9a80e000f43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:04.344963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b1dd3-cae0-43b1-af6b-e9a80e000f43 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=5989c8a1e28a43d3bc16970850ce49f4) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:04.345270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b1dd3-cae0-43b1-af6b-e9a80e000f43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:04.346740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b1dd3-cae0-43b1-af6b-e9a80e000f43 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:04.351983 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 634/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:26:04 2022] GET /identity/v3/role_assignments?user.id=5989c8a1e28a43d3bc16970850ce49f4&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:06.452275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e304e-14e7-45f1-9834-e0d21eb22e42 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:06.452775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e304e-14e7-45f1-9834-e0d21eb22e42 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:06.452888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e304e-14e7-45f1-9834-e0d21eb22e42 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:06.454169 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 627/1261] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:26:06.459309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b29c2-ee0e-44c8-a222-343d475acf93 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:06.459702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b29c2-ee0e-44c8-a222-343d475acf93 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:06.459866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b29c2-ee0e-44c8-a222-343d475acf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:06.479322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a69b29c2-ee0e-44c8-a222-343d475acf93 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:26:06.480514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a69b29c2-ee0e-44c8-a222-343d475acf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:26:06.545918 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 635/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:26:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:26:06.823480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e304e-14e7-45f1-9834-e0d21eb22e42 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:06.834215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:06.922183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e304e-14e7-45f1-9834-e0d21eb22e42 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:06.954798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819e5a9-1727-43dc-9271-ff11ce66118f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:06.955410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819e5a9-1727-43dc-9271-ff11ce66118f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:06.955683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819e5a9-1727-43dc-9271-ff11ce66118f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:06.955951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819e5a9-1727-43dc-9271-ff11ce66118f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:06.966148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819e5a9-1727-43dc-9271-ff11ce66118f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1819e5a9-1727-43dc-9271-ff11ce66118f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:26:06.970837 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:06.971903 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:06.972953 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 628/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:26:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:06.977936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b29c2-ee0e-44c8-a222-343d475acf93 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:06.995669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:07.090349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b29c2-ee0e-44c8-a222-343d475acf93 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:07.124277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:07.124874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:07.125159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:07.125429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:07.126114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:07.126417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:07.127784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:07.136392 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 636/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:26:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:07.142135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819e5a9-1727-43dc-9271-ff11ce66118f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:07.207687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:07.241121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819e5a9-1727-43dc-9271-ff11ce66118f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:07.274947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:07.276188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:07.276627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:07.276905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:07.286518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:07.287947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:26:07.302750 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:07.303858 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:26:07.305115 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:26:07.305115 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:07.305115 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:07.305115 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 629/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:07.309579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:07.389199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:07.410198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112f0a1-a187-45cd-9975-1b459dbedc8b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:07.442418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:07.443009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:07.443286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:07.443555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:07.444183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:07.444481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:07.445890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:07.457690 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 637/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:07.463402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:07.561325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842901da-1b06-402d-9081-2419c71b25b8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:07.579816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:07.593644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:07.594280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:07.594554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:07.594826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:07.601856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:07.603280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:26:07.614421 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:07.615542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:07.616510 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 630/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:07.621320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:07.716512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd108b3-77da-40e6-9209-1f134fcc89e8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:07.749461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:07.750071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:07.750350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:07.750619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:07.751328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:07.751629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:07.753047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:07.761434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:07.770058 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 638/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 30 09:26:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:07.776296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:07.872400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d4396-6391-4f80-88e5-0cc73e4c4c29 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:07.904422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:07.905029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:07.905305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:07.905573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:07.915596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:07.917582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:26:07.931080 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:07.932338 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:26:07.933314 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:26:07.933314 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:07.933314 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:07.933314 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 631/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:07.937781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:08.038110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a373d-0489-4e9f-9a3c-1174c70d2457 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:08.071469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:08.072060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:08.073060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:08.073300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:08.073961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:08.074260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:08.075621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:08.086314 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 639/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:08.092465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:08.124724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:08.198942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e250a-042c-4706-b0cf-1c329fe53cad None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:08.231588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:08.232199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:08.232474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:08.232743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:08.239065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:08.240493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:26:08.251832 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:08.252857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:08.258551 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 632/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:26:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:08.259148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:08.316765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:08.359607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903f177-5fb2-4846-9072-8636f39d5f00 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:08.393056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29987694-6c13-49de-9931-b8e090392870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:08.393689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29987694-6c13-49de-9931-b8e090392870 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:08.393971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29987694-6c13-49de-9931-b8e090392870 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:08.394242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29987694-6c13-49de-9931-b8e090392870 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:08.394933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29987694-6c13-49de-9931-b8e090392870 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:08.395240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29987694-6c13-49de-9931-b8e090392870 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:08.396715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29987694-6c13-49de-9931-b8e090392870 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:08.408883 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 640/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:08.414753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:08.505641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:08.514383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c7100-19c0-4014-9327-f592e8cb4262 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:08.546706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f6b60-2a4d-4c13-9795-bfffe3cba818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:08.547350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f6b60-2a4d-4c13-9795-bfffe3cba818 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:08.547650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f6b60-2a4d-4c13-9795-bfffe3cba818 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:08.547921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f6b60-2a4d-4c13-9795-bfffe3cba818 None admin] PATH_INFO: `/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/5989c8a1e28a43d3bc16970850ce49f4/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:08.549787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f6b60-2a4d-4c13-9795-bfffe3cba818 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2bf7840c042eeaf27440952d4aa8f, user_id=5989c8a1e28a43d3bc16970850ce49f4, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:08.551201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f6b60-2a4d-4c13-9795-bfffe3cba818 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:08.563731 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 633/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 30 09:26:08 2022] PUT /identity/v3/projects/e3b2bf7840c042eeaf27440952d4aa8f/users/5989c8a1e28a43d3bc16970850ce49f4/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:26:10.709005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7bfc8-c85b-4d18-bf96-962cf4786fd4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:10.709543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7bfc8-c85b-4d18-bf96-962cf4786fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:10.709543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7bfc8-c85b-4d18-bf96-962cf4786fd4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:10.710829 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 641/1274] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:26:10.716800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1420a5-2eaf-4346-841f-e4fb51585cee None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:10.717184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1420a5-2eaf-4346-841f-e4fb51585cee None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:10.717345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1420a5-2eaf-4346-841f-e4fb51585cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:10.735047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8f1420a5-2eaf-4346-841f-e4fb51585cee None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:26:10.736213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8f1420a5-2eaf-4346-841f-e4fb51585cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:26:10.799418 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 634/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:26:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:26:11.068978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7bfc8-c85b-4d18-bf96-962cf4786fd4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:11.173613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7bfc8-c85b-4d18-bf96-962cf4786fd4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:11.190140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:11.207067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b636ef-65a9-42e8-a3ba-36905d88189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:11.207683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b636ef-65a9-42e8-a3ba-36905d88189e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:11.207963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b636ef-65a9-42e8-a3ba-36905d88189e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:11.208234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b636ef-65a9-42e8-a3ba-36905d88189e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:11.219222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b636ef-65a9-42e8-a3ba-36905d88189e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-53b636ef-65a9-42e8-a3ba-36905d88189e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 30 09:26:11.223939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:11.225044 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:11.225957 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 642/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 30 09:26:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:11.231324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1420a5-2eaf-4346-841f-e4fb51585cee None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:11.308250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:11.326959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1420a5-2eaf-4346-841f-e4fb51585cee None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:11.359773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:11.360373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:11.360656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:11.360926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:11.361591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:11.361926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:11.363284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:11.371528 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 635/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 30 09:26:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:11.377326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b636ef-65a9-42e8-a3ba-36905d88189e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:11.477511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b636ef-65a9-42e8-a3ba-36905d88189e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:11.511522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:11.512149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:11.512427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:11.512697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:11.522565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:11.524287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:26:11.538915 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:11.540090 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:26:11.541213 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:26:11.541213 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:11.541213 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:11.541213 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 643/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:11.545820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:11.642048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de1903-bbea-4e11-82f6-469130c70236 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:11.674629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:11.675223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:11.675503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:11.675781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:11.676406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:11.676707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:11.678093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:11.690029 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 636/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:26:11.695741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:11.725869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:11.792963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4677d7-c0f3-4835-8783-ac92c6229604 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:11.824984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:11.825593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:11.825905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:11.826174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:11.833184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:11.834662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 30 09:26:11.845932 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:11.847025 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:11.848364 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 644/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:11.852722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:11.857087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:11.948245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c14484-8268-4944-938f-87616ef55998 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:11.980363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:11.980948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:11.981229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:11.981495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:11.982237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:11.982533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:11.983933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:12.000629 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 637/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 30 09:26:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:12.006293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:12.098013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:12.108288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d1c5-6754-4427-937f-0ee5d4eb7ecc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:12.141985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8879ad-e143-4f84-a672-89c81d76c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:12.142575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8879ad-e143-4f84-a672-89c81d76c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:12.142850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8879ad-e143-4f84-a672-89c81d76c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:12.143114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8879ad-e143-4f84-a672-89c81d76c1c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:12.206892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8879ad-e143-4f84-a672-89c81d76c1c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:12.208271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8879ad-e143-4f84-a672-89c81d76c1c4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:12.292085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:12.302413 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 645/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 30 09:26:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 30 09:26:12.308583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:12.368000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:12.409017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a954-28c0-4948-a463-37e9661e40b1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:12.442656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:12.443271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:12.443546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:12.443814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:12.454120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:12.455540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 30 09:26:12.469428 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:12.470550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 30 09:26:12.471476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 30 09:26:12.471476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 30 09:26:12.471476 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:12.471476 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 638/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:12.476262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8879ad-e143-4f84-a672-89c81d76c1c4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:12.572668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8879ad-e143-4f84-a672-89c81d76c1c4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:12.605428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:12.606057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:12.606342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:12.606616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:12.607254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:12.607559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:12.608936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:12.620631 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 646/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 30 09:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:12.626269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:12.722105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a09bdf-a264-40dc-9f95-8e2d65072c76 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:12.735487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:12.755293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:12.755906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:12.756181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:12.756449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:12.762950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:12.764370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 30 09:26:12.775419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:12.776683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Sep 30 09:26:12.777938 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 639/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 30 09:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:12.782359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:12.830926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:12.879808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d897042-7261-465e-9169-051728c2bf32 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:12.914990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f83c7a-7bb2-44df-aaf0-5f8cb19214f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:12.915587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f83c7a-7bb2-44df-aaf0-5f8cb19214f4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:12.916369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f83c7a-7bb2-44df-aaf0-5f8cb19214f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:12.916639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f83c7a-7bb2-44df-aaf0-5f8cb19214f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:12.917319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f83c7a-7bb2-44df-aaf0-5f8cb19214f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:12.917616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f83c7a-7bb2-44df-aaf0-5f8cb19214f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:12.919180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f83c7a-7bb2-44df-aaf0-5f8cb19214f4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:12.930663 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 647/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 30 09:26:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:12.937641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:13.034909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61402ff2-068c-45d1-b615-582578c9f10c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:13.067757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265e4f1-b9f6-4b80-9b1b-2406400bfee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:13.068347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265e4f1-b9f6-4b80-9b1b-2406400bfee3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:13.068622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265e4f1-b9f6-4b80-9b1b-2406400bfee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:13.068892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265e4f1-b9f6-4b80-9b1b-2406400bfee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:13.069600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265e4f1-b9f6-4b80-9b1b-2406400bfee3 None admin] RBAC: Adding query filter params (role.id=7beac02fae504326b332fd7fbdf80d35, scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f, user.id=5989c8a1e28a43d3bc16970850ce49f4) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:13.069938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265e4f1-b9f6-4b80-9b1b-2406400bfee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:13.071355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265e4f1-b9f6-4b80-9b1b-2406400bfee3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:13.077671 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 640/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 30 09:26:12 2022] GET /identity/v3/role_assignments?user.id=5989c8a1e28a43d3bc16970850ce49f4&scope.project.id=e3b2bf7840c042eeaf27440952d4aa8f&role.id=7beac02fae504326b332fd7fbdf80d35 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:15.223580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e458-d32f-43c4-a9f5-fefa7a4a1f15 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:15.224166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e458-d32f-43c4-a9f5-fefa7a4a1f15 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:15.224166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e458-d32f-43c4-a9f5-fefa7a4a1f15 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:15.225463 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 648/1288] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:26:15.231158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba405c-e0fe-406b-b3b7-56fb710e98c3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:15.231555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba405c-e0fe-406b-b3b7-56fb710e98c3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:15.231711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba405c-e0fe-406b-b3b7-56fb710e98c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:15.250109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-29ba405c-e0fe-406b-b3b7-56fb710e98c3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:26:15.251299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-29ba405c-e0fe-406b-b3b7-56fb710e98c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:26:15.279796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:15.316750 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 641/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:26:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:26:15.587315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864e458-d32f-43c4-a9f5-fefa7a4a1f15 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:15.685220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864e458-d32f-43c4-a9f5-fefa7a4a1f15 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:15.695432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:15.718582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:15.719193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:15.719470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:15.719739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] PATH_INFO: `/v3/services/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:15.720303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:15.721685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:15.730646 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:15.731858 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:15.732911 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 649/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:26:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:15.737815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba405c-e0fe-406b-b3b7-56fb710e98c3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:15.831028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba405c-e0fe-406b-b3b7-56fb710e98c3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:15.865720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b8eb5-d4df-4caa-826a-8826ad14002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:15.866344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8eb5-d4df-4caa-826a-8826ad14002b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:15.867571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8eb5-d4df-4caa-826a-8826ad14002b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:15.867843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8eb5-d4df-4caa-826a-8826ad14002b None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:15.868493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b8eb5-d4df-4caa-826a-8826ad14002b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:15.870156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b8eb5-d4df-4caa-826a-8826ad14002b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:15.872304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b8eb5-d4df-4caa-826a-8826ad14002b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:15.877667 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 642/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 30 09:26:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:15.883461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:15.980023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538ac32-be63-47e9-8ccc-c83b74508aab None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:16.012198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd58318-96df-4e6e-98b2-23cfb040f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:16.012804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd58318-96df-4e6e-98b2-23cfb040f483 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:16.013061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd58318-96df-4e6e-98b2-23cfb040f483 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:16.013326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd58318-96df-4e6e-98b2-23cfb040f483 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:16.014145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd58318-96df-4e6e-98b2-23cfb040f483 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:16.014470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd58318-96df-4e6e-98b2-23cfb040f483 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:16.015885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd58318-96df-4e6e-98b2-23cfb040f483 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:16.020480 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 650/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 30 09:26:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:18.157846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812f464-a7f5-4df0-932d-53eab2001021 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:18.158413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812f464-a7f5-4df0-932d-53eab2001021 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:18.158413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812f464-a7f5-4df0-932d-53eab2001021 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:18.159714 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 643/1293] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:26:18.164996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bdfe0-4987-4db6-bfb4-e83a82ea3d05 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:18.165396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bdfe0-4987-4db6-bfb4-e83a82ea3d05 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:18.165594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bdfe0-4987-4db6-bfb4-e83a82ea3d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:18.183810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-546bdfe0-4987-4db6-bfb4-e83a82ea3d05 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:26:18.185003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-546bdfe0-4987-4db6-bfb4-e83a82ea3d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:26:18.215665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:18.251756 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 651/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:26:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:26:18.521981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812f464-a7f5-4df0-932d-53eab2001021 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:18.553120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:18.621100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812f464-a7f5-4df0-932d-53eab2001021 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:18.655035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96b7dd-098b-4fb1-aaf9-31f38e61242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:18.655630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b7dd-098b-4fb1-aaf9-31f38e61242e None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:18.655909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b7dd-098b-4fb1-aaf9-31f38e61242e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:18.656180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96b7dd-098b-4fb1-aaf9-31f38e61242e None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:18.656831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b7dd-098b-4fb1-aaf9-31f38e61242e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:18.658225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96b7dd-098b-4fb1-aaf9-31f38e61242e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:18.672933 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 644/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 30 09:26:18 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:26:20.847919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338de98c-a620-4285-b125-b1477c9d3a97 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:20.848716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338de98c-a620-4285-b125-b1477c9d3a97 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:20.848716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338de98c-a620-4285-b125-b1477c9d3a97 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:20.850034 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 652/1296] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:26:20.856043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e224c7f-885f-4cdd-ad37-a900a4369e30 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:20.856506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e224c7f-885f-4cdd-ad37-a900a4369e30 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:20.857455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e224c7f-885f-4cdd-ad37-a900a4369e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:20.878293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8e224c7f-885f-4cdd-ad37-a900a4369e30 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:26:20.879367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e224c7f-885f-4cdd-ad37-a900a4369e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:26:20.961371 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 645/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:26:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:26:21.235878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338de98c-a620-4285-b125-b1477c9d3a97 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:21.333817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338de98c-a620-4285-b125-b1477c9d3a97 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:21.367194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:21.367807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:21.368084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:21.368351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:21.369551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:21.371020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:21.379603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:21.380819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:21.381764 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 653/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:26:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 30 09:26:21.387479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e224c7f-885f-4cdd-ad37-a900a4369e30 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:21.405599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:21.487488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e224c7f-885f-4cdd-ad37-a900a4369e30 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:21.520773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:21.521368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:21.521674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:21.521944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:21.522564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] RBAC: Adding query filter params (name=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:21.523114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:21.524464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:21.529542 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 646/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 30 09:26:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:21.535648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:21.566334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:21.632758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc760b-9363-4fb8-9878-3f6e2e14eb0c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:21.665313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbc293-01a8-45bb-a8ea-ce271def2caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:21.666103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbc293-01a8-45bb-a8ea-ce271def2caa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:21.666379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbc293-01a8-45bb-a8ea-ce271def2caa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:21.666650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbc293-01a8-45bb-a8ea-ce271def2caa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:21.667398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbc293-01a8-45bb-a8ea-ce271def2caa None admin] RBAC: Adding query filter params (interface=public, service_id=00ea25e10eee434697cd9040a100393b, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:21.667784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbc293-01a8-45bb-a8ea-ce271def2caa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:21.669344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbc293-01a8-45bb-a8ea-ce271def2caa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:21.674455 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 654/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 30 09:26:21 2022] GET /identity/v3/endpoints?service_id=00ea25e10eee434697cd9040a100393b&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:21.680049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:21.758572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:21.777199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd80d6-31db-43c2-ac28-476aacb7aefe None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:21.809523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0710b09-e5b4-442f-85c2-20109da97456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:21.810134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0710b09-e5b4-442f-85c2-20109da97456 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:21.810409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0710b09-e5b4-442f-85c2-20109da97456 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:21.810677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0710b09-e5b4-442f-85c2-20109da97456 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:21.811264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0710b09-e5b4-442f-85c2-20109da97456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:21.812611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0710b09-e5b4-442f-85c2-20109da97456 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:21.819110 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 647/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 30 09:26:21 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:26:23.974681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d83d7-ab2a-4104-9be3-85b79d082580 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:23.975237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d83d7-ab2a-4104-9be3-85b79d082580 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:23.975237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d83d7-ab2a-4104-9be3-85b79d082580 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:23.976562 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 655/1302] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:26:23.981899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf404b5-4fd2-4565-9192-56adebc2509b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:23.982296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf404b5-4fd2-4565-9192-56adebc2509b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:23.982454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf404b5-4fd2-4565-9192-56adebc2509b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:24.000496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ebf404b5-4fd2-4565-9192-56adebc2509b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:26:24.001695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ebf404b5-4fd2-4565-9192-56adebc2509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:26:24.069522 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 648/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:26:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 30 09:26:24.338026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d83d7-ab2a-4104-9be3-85b79d082580 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:24.433622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:24.440785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d83d7-ab2a-4104-9be3-85b79d082580 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:24.473118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:24.473765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:24.474042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:24.474308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:24.474879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:24.476240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 30 09:26:24.485159 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 30 09:26:24.486353 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Sep 30 09:26:24.487225 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 656/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 30 09:26:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 30 09:26:24.492436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf404b5-4fd2-4565-9192-56adebc2509b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:24.567719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:24.590153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf404b5-4fd2-4565-9192-56adebc2509b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:24.622006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1b6da-e895-414a-b74c-3f3cae8988a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:24.622602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1b6da-e895-414a-b74c-3f3cae8988a0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:24.622877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1b6da-e895-414a-b74c-3f3cae8988a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:24.623142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1b6da-e895-414a-b74c-3f3cae8988a0 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:24.623758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1b6da-e895-414a-b74c-3f3cae8988a0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:26:24.624057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1b6da-e895-414a-b74c-3f3cae8988a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:24.625398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1b6da-e895-414a-b74c-3f3cae8988a0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:24.630255 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 649/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 30 09:26:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:26:24.636687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:26:24.734499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8af0a-b8dd-43fc-8dd9-5cad6fcad2a0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:26:24.767295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290d420-4070-4be8-a2c9-67d05e6cbc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:26:24.767884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290d420-4070-4be8-a2c9-67d05e6cbc7e None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:26:24.768166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290d420-4070-4be8-a2c9-67d05e6cbc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:26:24.768441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290d420-4070-4be8-a2c9-67d05e6cbc7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:26:24.769096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290d420-4070-4be8-a2c9-67d05e6cbc7e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:26:24.770497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290d420-4070-4be8-a2c9-67d05e6cbc7e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:26:24.792829 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 657/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:26:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:26:34.627396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=30 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:26:34.788188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:29.173599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1bb7-6d9c-4cf6-a866-4b2fb2813164 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:29.174178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1bb7-6d9c-4cf6-a866-4b2fb2813164 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:29.174178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1bb7-6d9c-4cf6-a866-4b2fb2813164 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:29.181742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824748ec-4a82-4ff3-841d-acfaa4a2641f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:29.182214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824748ec-4a82-4ff3-841d-acfaa4a2641f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:29.182307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824748ec-4a82-4ff3-841d-acfaa4a2641f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:29.187033 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 650/1307] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:29.276136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-824748ec-4a82-4ff3-841d-acfaa4a2641f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:29.277554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-824748ec-4a82-4ff3-841d-acfaa4a2641f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:29.418452 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 658/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:27:29.716329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebfb55-8ad9-4d71-b84b-67a80554be0a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:29.716991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebfb55-8ad9-4d71-b84b-67a80554be0a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:29.717474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebfb55-8ad9-4d71-b84b-67a80554be0a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:29.719527 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 651/1309] 10.222.0.15 () {54 vars in 964 bytes} [Fri Sep 30 09:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:29.726711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406085a6-15cc-4f8a-b3e5-a2dbdcbf4040 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:29.726989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406085a6-15cc-4f8a-b3e5-a2dbdcbf4040 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:29.727257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406085a6-15cc-4f8a-b3e5-a2dbdcbf4040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:29.816210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-406085a6-15cc-4f8a-b3e5-a2dbdcbf4040 None None] MFA Rules not processed for user `f41e236339784d169258f06f1090b7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:29.817429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-406085a6-15cc-4f8a-b3e5-a2dbdcbf4040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:29.842058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:29.908129 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 659/1310] 10.222.0.15 () {58 vars in 1061 bytes} [Fri Sep 30 09:27:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:29.917744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebfb55-8ad9-4d71-b84b-67a80554be0a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:29.919826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-afebfb55-8ad9-4d71-b84b-67a80554be0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:30.044954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebfb55-8ad9-4d71-b84b-67a80554be0a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:30.127243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867935b2-da49-4d7a-a3c7-1e4519dbad4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:30.127776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867935b2-da49-4d7a-a3c7-1e4519dbad4f service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:30.128067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867935b2-da49-4d7a-a3c7-1e4519dbad4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:30.128344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867935b2-da49-4d7a-a3c7-1e4519dbad4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:30.129998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-867935b2-da49-4d7a-a3c7-1e4519dbad4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:30.188603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:30.249496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867935b2-da49-4d7a-a3c7-1e4519dbad4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:30.251140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867935b2-da49-4d7a-a3c7-1e4519dbad4f service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:30.273678 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 652/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:32.768424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dccbc-2fb7-450e-996d-3de2e90138b8 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:32.768931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dccbc-2fb7-450e-996d-3de2e90138b8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:32.768931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dccbc-2fb7-450e-996d-3de2e90138b8 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:32.770274 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 660/1312] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:32.776430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b0b41-eb14-4c74-b5be-8728e824533e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:32.776846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b0b41-eb14-4c74-b5be-8728e824533e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:32.776986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b0b41-eb14-4c74-b5be-8728e824533e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:32.862532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-500b0b41-eb14-4c74-b5be-8728e824533e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:32.863743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-500b0b41-eb14-4c74-b5be-8728e824533e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:32.965209 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 653/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:33.252874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dccbc-2fb7-450e-996d-3de2e90138b8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:33.254508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cc2dccbc-2fb7-450e-996d-3de2e90138b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:33.361325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dccbc-2fb7-450e-996d-3de2e90138b8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:33.440732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:33.443813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a4811-9458-441a-8833-f39b13d2af4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:33.444720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a4811-9458-441a-8833-f39b13d2af4b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:33.445261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a4811-9458-441a-8833-f39b13d2af4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:33.445796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a4811-9458-441a-8833-f39b13d2af4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:33.447779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-945a4811-9458-441a-8833-f39b13d2af4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:33.565202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a4811-9458-441a-8833-f39b13d2af4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:33.566712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a4811-9458-441a-8833-f39b13d2af4b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:33.586382 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 661/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:36.063174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5981ec-8b0b-4562-8b6d-fd3db14055e7 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:36.063728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5981ec-8b0b-4562-8b6d-fd3db14055e7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:36.063728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5981ec-8b0b-4562-8b6d-fd3db14055e7 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:36.065061 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 654/1315] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:36.070920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac320490-d659-4463-a497-a7c4bc000fd7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:36.071296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac320490-d659-4463-a497-a7c4bc000fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:36.071452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac320490-d659-4463-a497-a7c4bc000fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:36.136855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ac320490-d659-4463-a497-a7c4bc000fd7 None None] MFA Rules not processed for user `c28c6caabed44035add7bfb1c3325511`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:36.138085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ac320490-d659-4463-a497-a7c4bc000fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:36.188676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:36.233056 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 662/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:36.515763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5981ec-8b0b-4562-8b6d-fd3db14055e7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:36.518493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6a5981ec-8b0b-4562-8b6d-fd3db14055e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:36.530492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:36.622121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5981ec-8b0b-4562-8b6d-fd3db14055e7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:36.701941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6378da-2cd8-4fda-9ac1-4475f76ddf8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:36.702546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6378da-2cd8-4fda-9ac1-4475f76ddf8b service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:36.702823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6378da-2cd8-4fda-9ac1-4475f76ddf8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:36.703095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6378da-2cd8-4fda-9ac1-4475f76ddf8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:36.704682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1e6378da-2cd8-4fda-9ac1-4475f76ddf8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:36.767438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:36.815153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6378da-2cd8-4fda-9ac1-4475f76ddf8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:36.816576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6378da-2cd8-4fda-9ac1-4475f76ddf8b service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:36.836213 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 655/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:39.552587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1bd6b-8cc1-4b3c-8d16-6a2afe214b78 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:39.553121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1bd6b-8cc1-4b3c-8d16-6a2afe214b78 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:39.553121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1bd6b-8cc1-4b3c-8d16-6a2afe214b78 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:39.554495 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 663/1318] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:39.559685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1de5ff-426f-402d-bdf3-ce55c49444a0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:39.560073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1de5ff-426f-402d-bdf3-ce55c49444a0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:39.560226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1de5ff-426f-402d-bdf3-ce55c49444a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:39.624788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0b1de5ff-426f-402d-bdf3-ce55c49444a0 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:39.626009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0b1de5ff-426f-402d-bdf3-ce55c49444a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:39.721140 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 656/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:40.003286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1bd6b-8cc1-4b3c-8d16-6a2afe214b78 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:40.004826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dac1bd6b-8cc1-4b3c-8d16-6a2afe214b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:40.108877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1bd6b-8cc1-4b3c-8d16-6a2afe214b78 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:40.181718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c1c0a-e2ea-4591-b6fa-b5ef24d55b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:40.182314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1c0a-e2ea-4591-b6fa-b5ef24d55b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:40.182617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1c0a-e2ea-4591-b6fa-b5ef24d55b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:40.182889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c1c0a-e2ea-4591-b6fa-b5ef24d55b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:40.184479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6b0c1c0a-e2ea-4591-b6fa-b5ef24d55b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:40.191166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:40.290969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c1c0a-e2ea-4591-b6fa-b5ef24d55b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:40.292328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c1c0a-e2ea-4591-b6fa-b5ef24d55b73 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:40.310163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 664/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:43.146884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eec2a-53fa-4e7d-a4a5-293de0d6f857 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:43.147424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eec2a-53fa-4e7d-a4a5-293de0d6f857 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:43.147534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eec2a-53fa-4e7d-a4a5-293de0d6f857 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:43.149039 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 657/1321] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:43.154691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805001f-37d5-4fe8-8e6f-2d73201879a2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:43.155229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805001f-37d5-4fe8-8e6f-2d73201879a2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:43.155229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805001f-37d5-4fe8-8e6f-2d73201879a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:43.230637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c805001f-37d5-4fe8-8e6f-2d73201879a2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:43.232246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c805001f-37d5-4fe8-8e6f-2d73201879a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:43.294428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:43.338438 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 665/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:43.626767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eec2a-53fa-4e7d-a4a5-293de0d6f857 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:43.628335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6f1eec2a-53fa-4e7d-a4a5-293de0d6f857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:43.650361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:43.736449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eec2a-53fa-4e7d-a4a5-293de0d6f857 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:43.811068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c21124-4a79-443f-aac0-f66f0ee8fbfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:43.811679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c21124-4a79-443f-aac0-f66f0ee8fbfa service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:43.811985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c21124-4a79-443f-aac0-f66f0ee8fbfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:43.812259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c21124-4a79-443f-aac0-f66f0ee8fbfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:43.813967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-51c21124-4a79-443f-aac0-f66f0ee8fbfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:43.887059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:43.924598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c21124-4a79-443f-aac0-f66f0ee8fbfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:43.926074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c21124-4a79-443f-aac0-f66f0ee8fbfa service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:43.944699 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 658/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:46.163251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f7a92-0ab0-4c81-a4cd-569bc71afd69 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:46.163783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f7a92-0ab0-4c81-a4cd-569bc71afd69 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:46.163783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f7a92-0ab0-4c81-a4cd-569bc71afd69 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:46.165304 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 666/1324] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:46.170423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b8e91-030b-438a-8bd5-59ef6b32d9e4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:46.170863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b8e91-030b-438a-8bd5-59ef6b32d9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:46.170969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b8e91-030b-438a-8bd5-59ef6b32d9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:46.238308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-756b8e91-030b-438a-8bd5-59ef6b32d9e4 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:46.240063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-756b8e91-030b-438a-8bd5-59ef6b32d9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:46.349504 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 659/1325] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:46.636247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f7a92-0ab0-4c81-a4cd-569bc71afd69 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:46.637913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-089f7a92-0ab0-4c81-a4cd-569bc71afd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:46.742476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f7a92-0ab0-4c81-a4cd-569bc71afd69 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:46.816311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde11a84-cf1f-4afd-abfd-927fddab2103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:46.816830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde11a84-cf1f-4afd-abfd-927fddab2103 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:46.817112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde11a84-cf1f-4afd-abfd-927fddab2103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:46.817412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde11a84-cf1f-4afd-abfd-927fddab2103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:46.819056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cde11a84-cf1f-4afd-abfd-927fddab2103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:46.836450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:46.930296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde11a84-cf1f-4afd-abfd-927fddab2103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:46.931744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde11a84-cf1f-4afd-abfd-927fddab2103 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:46.952531 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 667/1326] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:49.179383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b7092-5a94-4c94-8455-391f519a5e82 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:49.179936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b7092-5a94-4c94-8455-391f519a5e82 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:49.179936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b7092-5a94-4c94-8455-391f519a5e82 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:49.181244 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 660/1327] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:49.186539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b74a1-889d-4b71-8006-054b87c7b257 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:49.186941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b74a1-889d-4b71-8006-054b87c7b257 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:49.187120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b74a1-889d-4b71-8006-054b87c7b257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:49.252872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ff9b74a1-889d-4b71-8006-054b87c7b257 None None] MFA Rules not processed for user `c28c6caabed44035add7bfb1c3325511`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:49.254147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ff9b74a1-889d-4b71-8006-054b87c7b257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:49.321437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:49.358275 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 668/1328] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:49.643367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b7092-5a94-4c94-8455-391f519a5e82 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:49.644923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8c2b7092-5a94-4c94-8455-391f519a5e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:49.678910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:49.756023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b7092-5a94-4c94-8455-391f519a5e82 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:49.831159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bab5e-0f3c-4ef9-9907-d80566629648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:49.831681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bab5e-0f3c-4ef9-9907-d80566629648 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:49.831953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bab5e-0f3c-4ef9-9907-d80566629648 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:49.832219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bab5e-0f3c-4ef9-9907-d80566629648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:49.834947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-505bab5e-0f3c-4ef9-9907-d80566629648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:49.914288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:49.946671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bab5e-0f3c-4ef9-9907-d80566629648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:49.948107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bab5e-0f3c-4ef9-9907-d80566629648 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:49.966598 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 661/1329] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:52.357924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69a374-2c55-4cec-93d5-41273b98306d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:52.358469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69a374-2c55-4cec-93d5-41273b98306d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:52.358469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69a374-2c55-4cec-93d5-41273b98306d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:52.359868 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 669/1330] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:52.365653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f187-f4d2-4796-965f-582b55d3dbe1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:52.366080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f187-f4d2-4796-965f-582b55d3dbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:52.366211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f187-f4d2-4796-965f-582b55d3dbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:52.432400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f4b6f187-f4d2-4796-965f-582b55d3dbe1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:52.433613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f4b6f187-f4d2-4796-965f-582b55d3dbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:52.532087 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 662/1331] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:52.824394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69a374-2c55-4cec-93d5-41273b98306d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:52.825971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1a69a374-2c55-4cec-93d5-41273b98306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:52.937426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69a374-2c55-4cec-93d5-41273b98306d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:53.010133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aab56b-c0c4-4824-a398-4c72dbd1beb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:53.010736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aab56b-c0c4-4824-a398-4c72dbd1beb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:53.011016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aab56b-c0c4-4824-a398-4c72dbd1beb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:53.011287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aab56b-c0c4-4824-a398-4c72dbd1beb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:53.012881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-02aab56b-c0c4-4824-a398-4c72dbd1beb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:53.042810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:53.123371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aab56b-c0c4-4824-a398-4c72dbd1beb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:53.124854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aab56b-c0c4-4824-a398-4c72dbd1beb3 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:53.142877 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 670/1332] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:55.328129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365cd20-073d-4039-af16-d6a3b8167bdd None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:55.328740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365cd20-073d-4039-af16-d6a3b8167bdd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:55.328740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365cd20-073d-4039-af16-d6a3b8167bdd None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:55.330062 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 663/1333] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:55.335981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f88d-fee3-4a57-92a0-6f548067cfcc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:55.336432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f88d-fee3-4a57-92a0-6f548067cfcc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:55.336534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292f88d-fee3-4a57-92a0-6f548067cfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:55.402145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2292f88d-fee3-4a57-92a0-6f548067cfcc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:55.403526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2292f88d-fee3-4a57-92a0-6f548067cfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:55.477591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:55.499430 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 671/1334] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:55.782348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365cd20-073d-4039-af16-d6a3b8167bdd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:55.783881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3365cd20-073d-4039-af16-d6a3b8167bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:55.823499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:55.884968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365cd20-073d-4039-af16-d6a3b8167bdd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:55.962137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f300a-fb34-420e-b685-f40b1db9570c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:55.962739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f300a-fb34-420e-b685-f40b1db9570c service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:55.963011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f300a-fb34-420e-b685-f40b1db9570c service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:55.963277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f300a-fb34-420e-b685-f40b1db9570c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:55.965128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5e0f300a-fb34-420e-b685-f40b1db9570c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:56.047709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:56.072889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f300a-fb34-420e-b685-f40b1db9570c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:56.074364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f300a-fb34-420e-b685-f40b1db9570c service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:56.092338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 664/1335] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:27:58.558205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee66fe3-74dd-42cf-83de-9dfa3639fec7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:58.558744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee66fe3-74dd-42cf-83de-9dfa3639fec7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:58.558744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee66fe3-74dd-42cf-83de-9dfa3639fec7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:58.560100 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 672/1336] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:27:58.565293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bec25-d0e2-4b32-a4ac-e139e492d888 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:58.565728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bec25-d0e2-4b32-a4ac-e139e492d888 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:58.565876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bec25-d0e2-4b32-a4ac-e139e492d888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:58.630765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5e8bec25-d0e2-4b32-a4ac-e139e492d888 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:27:58.631972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5e8bec25-d0e2-4b32-a4ac-e139e492d888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:58.728136 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 665/1337] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:27:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:27:59.007606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee66fe3-74dd-42cf-83de-9dfa3639fec7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:27:59.009164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6ee66fe3-74dd-42cf-83de-9dfa3639fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:59.115315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee66fe3-74dd-42cf-83de-9dfa3639fec7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:27:59.188878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be292a0-e564-484a-b64b-ccca0274abdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:27:59.189477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be292a0-e564-484a-b64b-ccca0274abdf service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:27:59.189789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be292a0-e564-484a-b64b-ccca0274abdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:27:59.190064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be292a0-e564-484a-b64b-ccca0274abdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:27:59.191659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5be292a0-e564-484a-b64b-ccca0274abdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:27:59.230172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:27:59.304069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be292a0-e564-484a-b64b-ccca0274abdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:27:59.305438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be292a0-e564-484a-b64b-ccca0274abdf service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:27:59.323662 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 673/1338] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:27:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:03.427139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f5a8d-c41e-43b2-98b5-996e53f7eac9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:03.428121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f5a8d-c41e-43b2-98b5-996e53f7eac9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:03.428395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f5a8d-c41e-43b2-98b5-996e53f7eac9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:03.430240 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 666/1339] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:03.436425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e7657-e261-46e9-8f96-256055daff42 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:03.436866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e7657-e261-46e9-8f96-256055daff42 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:03.436971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e7657-e261-46e9-8f96-256055daff42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:03.505203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9c5e7657-e261-46e9-8f96-256055daff42 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:03.506402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9c5e7657-e261-46e9-8f96-256055daff42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:03.602131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:03.617482 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 674/1340] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:03.903208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f5a8d-c41e-43b2-98b5-996e53f7eac9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:03.904796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-042f5a8d-c41e-43b2-98b5-996e53f7eac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:03.954039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:04.009078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f5a8d-c41e-43b2-98b5-996e53f7eac9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:04.085073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe9ef2-19a8-40f1-ac64-70e2d32904b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:04.085701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe9ef2-19a8-40f1-ac64-70e2d32904b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:04.085976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe9ef2-19a8-40f1-ac64-70e2d32904b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:04.086241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe9ef2-19a8-40f1-ac64-70e2d32904b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:04.087830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f0fe9ef2-19a8-40f1-ac64-70e2d32904b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:04.179192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:04.195652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe9ef2-19a8-40f1-ac64-70e2d32904b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:04.197098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe9ef2-19a8-40f1-ac64-70e2d32904b8 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:04.215966 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 667/1341] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:06.998959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd5ad8-a376-4f29-82e2-12bf56a68b5a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:06.999528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd5ad8-a376-4f29-82e2-12bf56a68b5a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:06.999528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd5ad8-a376-4f29-82e2-12bf56a68b5a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:07.000840 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 675/1342] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:07.006661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9f765-dd88-4e9b-b549-b14f33d94692 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:07.007102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9f765-dd88-4e9b-b549-b14f33d94692 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:07.007239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9f765-dd88-4e9b-b549-b14f33d94692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:07.084453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f4d9f765-dd88-4e9b-b549-b14f33d94692 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:07.085679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f4d9f765-dd88-4e9b-b549-b14f33d94692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:07.180232 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 668/1343] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:07.484367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396371a-62b4-4f5a-9f25-6c121078cc69 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:07.484736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396371a-62b4-4f5a-9f25-6c121078cc69 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:07.485106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396371a-62b4-4f5a-9f25-6c121078cc69 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:07.486581 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 676/1344] 10.222.0.15 () {54 vars in 964 bytes} [Fri Sep 30 09:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:07.493334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41c6d6-8001-4725-bbaf-593e10c41a70 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:07.493732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41c6d6-8001-4725-bbaf-593e10c41a70 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:07.494080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41c6d6-8001-4725-bbaf-593e10c41a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:07.564798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:07.574179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4e41c6d6-8001-4725-bbaf-593e10c41a70 None None] MFA Rules not processed for user `f41e236339784d169258f06f1090b7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:07.575424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4e41c6d6-8001-4725-bbaf-593e10c41a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:07.659679 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 669/1345] 10.222.0.15 () {58 vars in 1061 bytes} [Fri Sep 30 09:28:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:07.669245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396371a-62b4-4f5a-9f25-6c121078cc69 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:07.670792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2396371a-62b4-4f5a-9f25-6c121078cc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:07.772847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396371a-62b4-4f5a-9f25-6c121078cc69 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:07.847463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fd09b-b983-47f6-aace-3904c80a9e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:07.848140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd09b-b983-47f6-aace-3904c80a9e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:07.848412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd09b-b983-47f6-aace-3904c80a9e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:07.848678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd09b-b983-47f6-aace-3904c80a9e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:07.850343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2b6fd09b-b983-47f6-aace-3904c80a9e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:07.899223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:07.975347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd09b-b983-47f6-aace-3904c80a9e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:07.976700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd09b-b983-47f6-aace-3904c80a9e49 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:08.016252 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 677/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:12.965697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a292d-5f4f-4e24-99ee-a385440b6416 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:12.966276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a292d-5f4f-4e24-99ee-a385440b6416 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:12.966276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a292d-5f4f-4e24-99ee-a385440b6416 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:12.967581 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 670/1347] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:12.973852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54749f69-944d-4629-b148-73c577855bfc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:12.974293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54749f69-944d-4629-b148-73c577855bfc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:12.974440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54749f69-944d-4629-b148-73c577855bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:13.042886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-54749f69-944d-4629-b148-73c577855bfc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:13.044111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-54749f69-944d-4629-b148-73c577855bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:13.138976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:13.141925 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 678/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:13.432030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a292d-5f4f-4e24-99ee-a385440b6416 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:13.433745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-644a292d-5f4f-4e24-99ee-a385440b6416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:13.540908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a292d-5f4f-4e24-99ee-a385440b6416 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:13.586647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:13.613729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995fa0c-2f77-4eeb-a1bd-bbe3772568d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:13.614442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995fa0c-2f77-4eeb-a1bd-bbe3772568d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:13.614719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995fa0c-2f77-4eeb-a1bd-bbe3772568d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:13.614988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995fa0c-2f77-4eeb-a1bd-bbe3772568d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:13.616587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2995fa0c-2f77-4eeb-a1bd-bbe3772568d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:13.735636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995fa0c-2f77-4eeb-a1bd-bbe3772568d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:13.737106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995fa0c-2f77-4eeb-a1bd-bbe3772568d8 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:13.755377 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 671/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:18.705959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfea63c-a042-4a2e-b9ec-e00cf11e80dd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:18.706891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfea63c-a042-4a2e-b9ec-e00cf11e80dd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:18.706891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfea63c-a042-4a2e-b9ec-e00cf11e80dd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:18.709204 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 679/1350] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:18.715528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ae531-7d22-42f4-b4ed-b36acd82c734 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:18.715967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ae531-7d22-42f4-b4ed-b36acd82c734 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:18.716271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ae531-7d22-42f4-b4ed-b36acd82c734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:18.819602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8d8ae531-7d22-42f4-b4ed-b36acd82c734 None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:18.821223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8d8ae531-7d22-42f4-b4ed-b36acd82c734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:18.842971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:18.991744 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 672/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:28:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:23.331972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d7cf-a5af-4732-a113-635d4783aafd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:23.333036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d7cf-a5af-4732-a113-635d4783aafd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:23.333535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74d7cf-a5af-4732-a113-635d4783aafd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:23.335946 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 680/1352] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:28:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:23.342552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85367e18-72c3-4b34-b010-4b13e378d7f5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:23.343360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85367e18-72c3-4b34-b010-4b13e378d7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:23.343360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85367e18-72c3-4b34-b010-4b13e378d7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:23.434780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-85367e18-72c3-4b34-b010-4b13e378d7f5 None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:23.436383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-85367e18-72c3-4b34-b010-4b13e378d7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:23.511439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:23.554738 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 673/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:28:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:24.825553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ff0af-5bb8-44a4-8b87-1133eec0173b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:24.826172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ff0af-5bb8-44a4-8b87-1133eec0173b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:24.826366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ff0af-5bb8-44a4-8b87-1133eec0173b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:24.827806 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 681/1354] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:24.833892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521851e-9479-43fc-90b9-a78b7da4c779 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:24.834346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521851e-9479-43fc-90b9-a78b7da4c779 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:24.834436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521851e-9479-43fc-90b9-a78b7da4c779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:24.912488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f521851e-9479-43fc-90b9-a78b7da4c779 None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:24.913800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f521851e-9479-43fc-90b9-a78b7da4c779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:25.019806 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 674/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:28:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:28.702672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a120-d733-4f30-b4d1-79715cae7b2f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:28.703257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a120-d733-4f30-b4d1-79715cae7b2f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:28.703494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a120-d733-4f30-b4d1-79715cae7b2f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:28.705048 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 682/1356] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:28.711566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd75a8-9e9f-4aaa-ba7c-f473c26e6d64 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:28.712031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd75a8-9e9f-4aaa-ba7c-f473c26e6d64 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:28.712136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd75a8-9e9f-4aaa-ba7c-f473c26e6d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:28.749440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:28.792274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f5cd75a8-9e9f-4aaa-ba7c-f473c26e6d64 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:28.793504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f5cd75a8-9e9f-4aaa-ba7c-f473c26e6d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:28.895013 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 675/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:29.203027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c36c48-7620-4517-84f2-4790e9a9d458 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:29.203793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c36c48-7620-4517-84f2-4790e9a9d458 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:29.204366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c36c48-7620-4517-84f2-4790e9a9d458 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:29.209255 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 683/1358] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 30 09:28:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:29.223578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e43a-62b8-48d9-a911-e8121e6d4afa None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:29.224341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e43a-62b8-48d9-a911-e8121e6d4afa None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:29.224980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e43a-62b8-48d9-a911-e8121e6d4afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:29.306336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1a85e43a-62b8-48d9-a911-e8121e6d4afa None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:29.307652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1a85e43a-62b8-48d9-a911-e8121e6d4afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:29.360169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:29.448869 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 676/1359] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 30 09:28:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:29.457268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c36c48-7620-4517-84f2-4790e9a9d458 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:29.458914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e2c36c48-7620-4517-84f2-4790e9a9d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:29.616027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c36c48-7620-4517-84f2-4790e9a9d458 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:29.692504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:29.734010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dba56c-7539-4ad0-8093-5e3d1afcd84b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:29.734666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dba56c-7539-4ad0-8093-5e3d1afcd84b service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:29.734930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dba56c-7539-4ad0-8093-5e3d1afcd84b service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:29.735192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dba56c-7539-4ad0-8093-5e3d1afcd84b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:29.736795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-83dba56c-7539-4ad0-8093-5e3d1afcd84b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:29.838422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dba56c-7539-4ad0-8093-5e3d1afcd84b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:29.840052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dba56c-7539-4ad0-8093-5e3d1afcd84b service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:29.859401 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 684/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:28:32.325444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806061a0-089d-4b64-8ab4-319253076da9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:32.326088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806061a0-089d-4b64-8ab4-319253076da9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:32.326088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806061a0-089d-4b64-8ab4-319253076da9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:32.333852 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 677/1361] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:32.334072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda514bf-98b1-4be1-b85b-982809496587 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:32.334072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda514bf-98b1-4be1-b85b-982809496587 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:32.334072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda514bf-98b1-4be1-b85b-982809496587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:32.410374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fda514bf-98b1-4be1-b85b-982809496587 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:32.411603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fda514bf-98b1-4be1-b85b-982809496587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:32.452063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:32.516050 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 685/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:32.831752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806061a0-089d-4b64-8ab4-319253076da9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:32.833409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-806061a0-089d-4b64-8ab4-319253076da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:33.015211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806061a0-089d-4b64-8ab4-319253076da9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:33.027755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:33.151238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5442a-5f07-46d1-bf2d-0f55d51e19b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:33.151844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5442a-5f07-46d1-bf2d-0f55d51e19b9 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:33.152121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5442a-5f07-46d1-bf2d-0f55d51e19b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:33.152388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5442a-5f07-46d1-bf2d-0f55d51e19b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:33.154020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-65f5442a-5f07-46d1-bf2d-0f55d51e19b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:33.245271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5442a-5f07-46d1-bf2d-0f55d51e19b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:33.246871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5442a-5f07-46d1-bf2d-0f55d51e19b9 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:33.265419 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 678/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:35.798727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9cc2-4fd3-4ffb-9785-4ffe46ebbe40 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:35.799274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9cc2-4fd3-4ffb-9785-4ffe46ebbe40 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:35.799274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b9cc2-4fd3-4ffb-9785-4ffe46ebbe40 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:35.801458 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 686/1364] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:35.807024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e578b-414e-4f4f-bd7e-f2c233d84f49 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:35.807458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e578b-414e-4f4f-bd7e-f2c233d84f49 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:35.807632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e578b-414e-4f4f-bd7e-f2c233d84f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:35.824957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:35.879964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-754e578b-414e-4f4f-bd7e-f2c233d84f49 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:35.881316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-754e578b-414e-4f4f-bd7e-f2c233d84f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:35.985668 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 679/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:36.277356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed129c-f0e7-4a11-94b2-4f510ab5c05a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:36.277661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed129c-f0e7-4a11-94b2-4f510ab5c05a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:36.277943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed129c-f0e7-4a11-94b2-4f510ab5c05a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:36.279427 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 687/1366] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 30 09:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:28:36.285579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6d633-705e-4278-b77f-60244a3805d7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:36.286141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6d633-705e-4278-b77f-60244a3805d7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:36.286513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6d633-705e-4278-b77f-60244a3805d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:36.368440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-73a6d633-705e-4278-b77f-60244a3805d7 None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:36.369655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-73a6d633-705e-4278-b77f-60244a3805d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:36.412067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:36.498859 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 680/1367] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 30 09:28:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:36.507473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed129c-f0e7-4a11-94b2-4f510ab5c05a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:36.508992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c7ed129c-f0e7-4a11-94b2-4f510ab5c05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:36.661952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed129c-f0e7-4a11-94b2-4f510ab5c05a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:36.683277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:36.784534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e72b9-4d4b-4a2e-8526-8566ffab225b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:36.785056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e72b9-4d4b-4a2e-8526-8566ffab225b service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:36.785331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e72b9-4d4b-4a2e-8526-8566ffab225b service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:36.785603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e72b9-4d4b-4a2e-8526-8566ffab225b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:36.787289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-333e72b9-4d4b-4a2e-8526-8566ffab225b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:36.888931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e72b9-4d4b-4a2e-8526-8566ffab225b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:36.890571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e72b9-4d4b-4a2e-8526-8566ffab225b service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:36.912697 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 688/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:36.969880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d868e4-b0a3-41c5-8573-9001a39adbd2 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:36.970410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d868e4-b0a3-41c5-8573-9001a39adbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:36.970410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d868e4-b0a3-41c5-8573-9001a39adbd2 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:36.972568 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 681/1369] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:28:36.977748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03e64-8d09-4b9d-b77e-69d5a1de4340 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:36.978038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03e64-8d09-4b9d-b77e-69d5a1de4340 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:36.978311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e03e64-8d09-4b9d-b77e-69d5a1de4340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:37.016853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:37.060848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e2e03e64-8d09-4b9d-b77e-69d5a1de4340 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:37.062006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e2e03e64-8d09-4b9d-b77e-69d5a1de4340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:37.188689 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 689/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:37.524906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743239c7-9095-48fb-975e-6330e958c55d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:37.524906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743239c7-9095-48fb-975e-6330e958c55d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:37.524906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743239c7-9095-48fb-975e-6330e958c55d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:37.529204 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 682/1371] 10.222.0.15 () {54 vars in 963 bytes} [Fri Sep 30 09:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:28:37.531307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bac84-2ea9-4394-b169-f5ccbecf5842 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:37.531547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bac84-2ea9-4394-b169-f5ccbecf5842 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:37.531819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bac84-2ea9-4394-b169-f5ccbecf5842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:37.611323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-169bac84-2ea9-4394-b169-f5ccbecf5842 None None] MFA Rules not processed for user `2c1784d615d8439a9a4cb9fea305a41b`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:37.613375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-169bac84-2ea9-4394-b169-f5ccbecf5842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:37.665847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:37.732212 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 690/1372] 10.222.0.15 () {58 vars in 1060 bytes} [Fri Sep 30 09:28:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:37.733302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743239c7-9095-48fb-975e-6330e958c55d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:37.735218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-743239c7-9095-48fb-975e-6330e958c55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:37.882795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743239c7-9095-48fb-975e-6330e958c55d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:37.936654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:37.985437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee24ae-5ddd-4603-955a-06f4c5f376c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:37.986120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee24ae-5ddd-4603-955a-06f4c5f376c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:37.986346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee24ae-5ddd-4603-955a-06f4c5f376c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:37.986648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee24ae-5ddd-4603-955a-06f4c5f376c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:37.988765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-08ee24ae-5ddd-4603-955a-06f4c5f376c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:38.160871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee24ae-5ddd-4603-955a-06f4c5f376c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:38.162303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee24ae-5ddd-4603-955a-06f4c5f376c1 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:38.181366 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 683/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:39.226307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603eab5-bb66-426e-89e5-4e6b5d278619 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:39.226869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603eab5-bb66-426e-89e5-4e6b5d278619 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:39.226869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603eab5-bb66-426e-89e5-4e6b5d278619 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:39.228212 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 691/1374] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:39.233422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521224-bed5-401c-baa8-07d2a61f7cc6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:39.233848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521224-bed5-401c-baa8-07d2a61f7cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:39.234044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521224-bed5-401c-baa8-07d2a61f7cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:39.301782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-38521224-bed5-401c-baa8-07d2a61f7cc6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:39.302979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-38521224-bed5-401c-baa8-07d2a61f7cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:39.323473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:39.400655 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 684/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:39.702187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603eab5-bb66-426e-89e5-4e6b5d278619 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:39.703766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e603eab5-bb66-426e-89e5-4e6b5d278619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:39.862482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603eab5-bb66-426e-89e5-4e6b5d278619 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:39.916953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:39.980729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93392e6-9515-46ca-b106-0fa8d9a23ba5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:39.981341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93392e6-9515-46ca-b106-0fa8d9a23ba5 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:39.981643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93392e6-9515-46ca-b106-0fa8d9a23ba5 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:39.981926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93392e6-9515-46ca-b106-0fa8d9a23ba5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:39.983527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a93392e6-9515-46ca-b106-0fa8d9a23ba5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:40.077859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93392e6-9515-46ca-b106-0fa8d9a23ba5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:40.079492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93392e6-9515-46ca-b106-0fa8d9a23ba5 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:40.097885 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 692/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:42.342444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129114fd-0ce1-4e02-9674-47f5ce5b3d94 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:42.342972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129114fd-0ce1-4e02-9674-47f5ce5b3d94 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:42.342972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129114fd-0ce1-4e02-9674-47f5ce5b3d94 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:42.344463 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 685/1377] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:42.350336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95024303-3a2c-4928-842b-1b021ef004c2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:42.350767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95024303-3a2c-4928-842b-1b021ef004c2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:42.350933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95024303-3a2c-4928-842b-1b021ef004c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:42.421222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-95024303-3a2c-4928-842b-1b021ef004c2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:42.422448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-95024303-3a2c-4928-842b-1b021ef004c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:42.429829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:42.542640 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 693/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:42.829263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129114fd-0ce1-4e02-9674-47f5ce5b3d94 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:42.831707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-129114fd-0ce1-4e02-9674-47f5ce5b3d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:43.003558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129114fd-0ce1-4e02-9674-47f5ce5b3d94 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:43.007852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:43.129116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04e8b5-099d-47f5-a7ea-a6e51c579a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:43.130547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04e8b5-099d-47f5-a7ea-a6e51c579a5f service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:43.130936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04e8b5-099d-47f5-a7ea-a6e51c579a5f service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:43.131308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04e8b5-099d-47f5-a7ea-a6e51c579a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:43.133079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6b04e8b5-099d-47f5-a7ea-a6e51c579a5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:43.232493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04e8b5-099d-47f5-a7ea-a6e51c579a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:43.234369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04e8b5-099d-47f5-a7ea-a6e51c579a5f service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:43.254030 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 686/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:45.474337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46aaa8-3948-4912-8ae8-dc9e4aec3c5b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:45.474856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46aaa8-3948-4912-8ae8-dc9e4aec3c5b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:45.474937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46aaa8-3948-4912-8ae8-dc9e4aec3c5b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:45.476378 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 694/1380] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:28:45.481516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebef1-fe0c-41c7-b8a4-245fbb69832d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:45.482035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebef1-fe0c-41c7-b8a4-245fbb69832d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:45.482138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebef1-fe0c-41c7-b8a4-245fbb69832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:45.494162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:45.552497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ca9ebef1-fe0c-41c7-b8a4-245fbb69832d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:45.553605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ca9ebef1-fe0c-41c7-b8a4-245fbb69832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:45.720113 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 687/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:46.011349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46aaa8-3948-4912-8ae8-dc9e4aec3c5b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:46.012999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8b46aaa8-3948-4912-8ae8-dc9e4aec3c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:46.153674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:46.165336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46aaa8-3948-4912-8ae8-dc9e4aec3c5b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:46.280643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4fdc1-5fa8-4f5b-91fd-23f235b12376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:46.281195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4fdc1-5fa8-4f5b-91fd-23f235b12376 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:46.281487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4fdc1-5fa8-4f5b-91fd-23f235b12376 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:46.281844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4fdc1-5fa8-4f5b-91fd-23f235b12376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:46.283436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-95b4fdc1-5fa8-4f5b-91fd-23f235b12376 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:46.379291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4fdc1-5fa8-4f5b-91fd-23f235b12376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:46.381241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4fdc1-5fa8-4f5b-91fd-23f235b12376 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:46.392466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:46.403842 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 695/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:48.728203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9444-dbaa-44ef-8d51-4065caecdd59 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:48.728845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9444-dbaa-44ef-8d51-4065caecdd59 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:48.728845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9444-dbaa-44ef-8d51-4065caecdd59 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:48.730669 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 688/1383] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:48.735782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c8ad-3029-4934-904a-429cca1bb232 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:48.736182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c8ad-3029-4934-904a-429cca1bb232 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:48.736333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c8ad-3029-4934-904a-429cca1bb232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:48.808283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f8c4c8ad-3029-4934-904a-429cca1bb232 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:48.809495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f8c4c8ad-3029-4934-904a-429cca1bb232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:48.910622 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 696/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:49.210093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9444-dbaa-44ef-8d51-4065caecdd59 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:49.211609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3dfc9444-dbaa-44ef-8d51-4065caecdd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:49.303495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:49.368709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9444-dbaa-44ef-8d51-4065caecdd59 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:49.491976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d4923-f012-4cd1-8461-8151ec0db33c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:49.492547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d4923-f012-4cd1-8461-8151ec0db33c service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:49.492850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d4923-f012-4cd1-8461-8151ec0db33c service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:49.493123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d4923-f012-4cd1-8461-8151ec0db33c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:49.494813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fe7d4923-f012-4cd1-8461-8151ec0db33c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:49.549245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:49.594042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d4923-f012-4cd1-8461-8151ec0db33c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:49.595820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d4923-f012-4cd1-8461-8151ec0db33c service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:49.616530 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 689/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:51.849088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ed2c2-b94d-4d5c-a923-be4a990d92fd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:51.849734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ed2c2-b94d-4d5c-a923-be4a990d92fd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:51.849734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ed2c2-b94d-4d5c-a923-be4a990d92fd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:51.850931 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 697/1386] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:51.856142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2498e0-2e65-4b45-8592-3647a96f869b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:51.856684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2498e0-2e65-4b45-8592-3647a96f869b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:51.856684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2498e0-2e65-4b45-8592-3647a96f869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:51.927622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6c2498e0-2e65-4b45-8592-3647a96f869b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:51.929020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6c2498e0-2e65-4b45-8592-3647a96f869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:52.037550 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 690/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:52.373011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ed2c2-b94d-4d5c-a923-be4a990d92fd None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:52.374635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-616ed2c2-b94d-4d5c-a923-be4a990d92fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:52.463341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:52.534341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ed2c2-b94d-4d5c-a923-be4a990d92fd None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:52.652666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e120c-1fc9-42d1-81d0-153fc8200816 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:52.653309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e120c-1fc9-42d1-81d0-153fc8200816 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:52.653635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e120c-1fc9-42d1-81d0-153fc8200816 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:52.653994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e120c-1fc9-42d1-81d0-153fc8200816 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:52.656043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f76e120c-1fc9-42d1-81d0-153fc8200816 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:52.701080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:52.759764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e120c-1fc9-42d1-81d0-153fc8200816 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:52.761795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e120c-1fc9-42d1-81d0-153fc8200816 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:52.783425 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 698/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:55.210195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55026f-c091-4ed3-b70d-70339e90e50f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:55.211027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55026f-c091-4ed3-b70d-70339e90e50f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:55.211027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55026f-c091-4ed3-b70d-70339e90e50f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:55.212100 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 691/1389] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:55.217546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da686335-3e32-4002-9280-093e893fec81 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:55.218154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da686335-3e32-4002-9280-093e893fec81 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:55.218376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da686335-3e32-4002-9280-093e893fec81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:55.292756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-da686335-3e32-4002-9280-093e893fec81 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:55.294015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-da686335-3e32-4002-9280-093e893fec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:55.396843 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 699/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:55.735037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55026f-c091-4ed3-b70d-70339e90e50f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:55.736579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6b55026f-c091-4ed3-b70d-70339e90e50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:55.768980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:55.902573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55026f-c091-4ed3-b70d-70339e90e50f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:56.019606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:56.032617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594da96-a351-4917-ab43-bf5585374984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:56.034193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594da96-a351-4917-ab43-bf5585374984 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:56.034609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594da96-a351-4917-ab43-bf5585374984 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:56.034992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594da96-a351-4917-ab43-bf5585374984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:56.037379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6594da96-a351-4917-ab43-bf5585374984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:56.139651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594da96-a351-4917-ab43-bf5585374984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:56.141274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594da96-a351-4917-ab43-bf5585374984 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:56.161031 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 692/1391] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:58.363487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea73238-a87d-4c93-a059-16023b2968c6 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:58.364058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea73238-a87d-4c93-a059-16023b2968c6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:58.364058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea73238-a87d-4c93-a059-16023b2968c6 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:58.365402 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 700/1392] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:28:58.370825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca0fb1-64a9-4867-a551-9e113e7a6e9b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:58.371723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca0fb1-64a9-4867-a551-9e113e7a6e9b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:58.372006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca0fb1-64a9-4867-a551-9e113e7a6e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:58.442254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3cca0fb1-64a9-4867-a551-9e113e7a6e9b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:58.443451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3cca0fb1-64a9-4867-a551-9e113e7a6e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:58.503279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:58.545683 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 693/1393] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:58.669832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ed90a-1145-4a5c-b7d8-73c582e171e3 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:58.670236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ed90a-1145-4a5c-b7d8-73c582e171e3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:58.670523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ed90a-1145-4a5c-b7d8-73c582e171e3 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:58.671869 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 701/1394] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:28:58.678265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434d445-0cf5-4b97-939a-fde382701be2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:58.678555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434d445-0cf5-4b97-939a-fde382701be2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:58.678832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434d445-0cf5-4b97-939a-fde382701be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:58.747598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c434d445-0cf5-4b97-939a-fde382701be2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:58.748808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c434d445-0cf5-4b97-939a-fde382701be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:58.834936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ed90a-1145-4a5c-b7d8-73c582e171e3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:58.836450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bc3ed90a-1145-4a5c-b7d8-73c582e171e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:58.845852 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 694/1395] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:58.850994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:58.984664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ed90a-1145-4a5c-b7d8-73c582e171e3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:59.075626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:59.105881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:59.106492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:59.106777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:59.107048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:59.108647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:59.158003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f1cf-41bc-4e3f-81bf-f0c2e842f3d2 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:59.159357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f1cf-41bc-4e3f-81bf-f0c2e842f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:59.159663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f1cf-41bc-4e3f-81bf-f0c2e842f3d2 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:59.161571 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 695/1396] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 30 09:28:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:28:59.166992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367b10b-d1e5-4328-9194-48d0d03567b1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:59.167272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367b10b-d1e5-4328-9194-48d0d03567b1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:59.167547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367b10b-d1e5-4328-9194-48d0d03567b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:59.203885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:59.205575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:59.223764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:59.224200 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 702/1397] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:28:59.247126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2367b10b-d1e5-4328-9194-48d0d03567b1 None None] MFA Rules not processed for user `7fe7a090bd7941bb827d301aa4949857`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:28:59.248342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2367b10b-d1e5-4328-9194-48d0d03567b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:59.329182 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 696/1398] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 30 09:28:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:28:59.339200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:28:59.340595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:59.434502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b93a30-7e56-4798-9fc4-fd01fe33df6d service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:28:59.464201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:28:59.512196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc66d4-3a50-4a8a-8a6c-920694b76954 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fe7a090bd7941bb827d301aa4949857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:28:59.512808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc66d4-3a50-4a8a-8a6c-920694b76954 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:28:59.513089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc66d4-3a50-4a8a-8a6c-920694b76954 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:28:59.513361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc66d4-3a50-4a8a-8a6c-920694b76954 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:28:59.514998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d8fc66d4-3a50-4a8a-8a6c-920694b76954 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:28:59.625678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc66d4-3a50-4a8a-8a6c-920694b76954 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:28:59.627162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc66d4-3a50-4a8a-8a6c-920694b76954 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:28:59.645758 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 703/1399] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:00.409000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d67e0-b78e-42ce-8d28-38baf9d2cd07 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:00.409534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d67e0-b78e-42ce-8d28-38baf9d2cd07 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:00.409781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d67e0-b78e-42ce-8d28-38baf9d2cd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:00.477614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7d4d67e0-b78e-42ce-8d28-38baf9d2cd07 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:00.478851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7d4d67e0-b78e-42ce-8d28-38baf9d2cd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:00.568325 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 697/1400] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:29:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:00.614414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d751-2f0d-4aba-b635-58e6132cbd02 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:00.614864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d751-2f0d-4aba-b635-58e6132cbd02 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:00.615097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150d751-2f0d-4aba-b635-58e6132cbd02 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:00.616525 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 704/1401] 10.222.0.15 () {54 vars in 943 bytes} [Fri Sep 30 09:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:00.622959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab7b40-ab87-430d-b528-aae5edaccc82 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:00.623245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab7b40-ab87-430d-b528-aae5edaccc82 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:00.623520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab7b40-ab87-430d-b528-aae5edaccc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:00.628280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:00.701184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-16ab7b40-ab87-430d-b528-aae5edaccc82 None None] MFA Rules not processed for user `60761ed7c27f4018a6b4791b3b060bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:00.702406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-16ab7b40-ab87-430d-b528-aae5edaccc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:00.807367 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 698/1402] 10.222.0.15 () {58 vars in 1040 bytes} [Fri Sep 30 09:29:00 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:00.817605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150d751-2f0d-4aba-b635-58e6132cbd02 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:00.819161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d150d751-2f0d-4aba-b635-58e6132cbd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:00.885693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:00.964865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150d751-2f0d-4aba-b635-58e6132cbd02 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:01.072743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9412c-830f-4649-b1dd-809442738e64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:01.073349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9412c-830f-4649-b1dd-809442738e64 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:01.073654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9412c-830f-4649-b1dd-809442738e64 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:01.073924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9412c-830f-4649-b1dd-809442738e64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:01.075540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b6a9412c-830f-4649-b1dd-809442738e64 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:01.121087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:01.160314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9412c-830f-4649-b1dd-809442738e64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:01.161941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9412c-830f-4649-b1dd-809442738e64 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:01.169891 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 705/1403] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:29:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:29:01.470778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a105f-21b4-43ba-95eb-7d9513ecee94 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:01.471873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a105f-21b4-43ba-95eb-7d9513ecee94 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:01.472092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a105f-21b4-43ba-95eb-7d9513ecee94 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:01.473589 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 699/1404] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:29:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:01.478709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc30d-7ec9-4cac-a39f-6f288e5db32c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:01.479001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc30d-7ec9-4cac-a39f-6f288e5db32c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:01.479277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc30d-7ec9-4cac-a39f-6f288e5db32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:01.551057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6cdfc30d-7ec9-4cac-a39f-6f288e5db32c None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:01.552323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6cdfc30d-7ec9-4cac-a39f-6f288e5db32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:01.657064 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 706/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:01.951655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a105f-21b4-43ba-95eb-7d9513ecee94 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:01.953180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-145a105f-21b4-43ba-95eb-7d9513ecee94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:02.033220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:02.131570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a105f-21b4-43ba-95eb-7d9513ecee94 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:02.263759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc3883-a758-40ce-9b74-ef62ee01a95f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:02.264496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3883-a758-40ce-9b74-ef62ee01a95f service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:02.264773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3883-a758-40ce-9b74-ef62ee01a95f service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:02.265044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc3883-a758-40ce-9b74-ef62ee01a95f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:02.266679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-15fc3883-a758-40ce-9b74-ef62ee01a95f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:02.293039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:02.370063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc3883-a758-40ce-9b74-ef62ee01a95f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:02.371796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc3883-a758-40ce-9b74-ef62ee01a95f service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:02.391430 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 700/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:04.673843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008c60-e61e-4774-b9b8-17784d3f7d2a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:04.674468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008c60-e61e-4774-b9b8-17784d3f7d2a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:04.674468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e008c60-e61e-4774-b9b8-17784d3f7d2a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:04.676195 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 707/1407] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:04.681498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664cf6-0613-44bc-8c91-1777e38df16e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:04.681932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664cf6-0613-44bc-8c91-1777e38df16e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:04.682114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664cf6-0613-44bc-8c91-1777e38df16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:04.759070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e7664cf6-0613-44bc-8c91-1777e38df16e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:04.760318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e7664cf6-0613-44bc-8c91-1777e38df16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:04.866888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:04.869446 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 701/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:05.209584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008c60-e61e-4774-b9b8-17784d3f7d2a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:05.211432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2e008c60-e61e-4774-b9b8-17784d3f7d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:05.277156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:05.407314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e008c60-e61e-4774-b9b8-17784d3f7d2a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:05.532656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:05.535573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1bcb8d-f8c8-4253-a236-c83cdebb8b09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:05.537126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bcb8d-f8c8-4253-a236-c83cdebb8b09 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:05.537406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bcb8d-f8c8-4253-a236-c83cdebb8b09 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:05.537709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1bcb8d-f8c8-4253-a236-c83cdebb8b09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:05.539604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dd1bcb8d-f8c8-4253-a236-c83cdebb8b09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:05.640911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1bcb8d-f8c8-4253-a236-c83cdebb8b09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:05.642572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1bcb8d-f8c8-4253-a236-c83cdebb8b09 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:05.662469 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 708/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:15.659494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=46 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:24.286894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6bd5a-8417-47a3-89eb-4831529b76f1 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:24.287453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6bd5a-8417-47a3-89eb-4831529b76f1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:24.287723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6bd5a-8417-47a3-89eb-4831529b76f1 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:24.289076 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 702/1410] 10.222.0.15 () {54 vars in 918 bytes} [Fri Sep 30 09:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:29:24.294134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36e7b-5b0a-4a51-953f-2fac04cef0fc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:24.294544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36e7b-5b0a-4a51-953f-2fac04cef0fc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:24.294709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36e7b-5b0a-4a51-953f-2fac04cef0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:24.490262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dda36e7b-5b0a-4a51-953f-2fac04cef0fc None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:24.491548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dda36e7b-5b0a-4a51-953f-2fac04cef0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:24.599581 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 709/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 30 09:29:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:24.614640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c23d82-7c6e-426e-a37e-9c457da6a5d9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:24.614926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c23d82-7c6e-426e-a37e-9c457da6a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:24.615201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c23d82-7c6e-426e-a37e-9c457da6a5d9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:24.616588 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 703/1412] 10.222.0.15 () {54 vars in 918 bytes} [Fri Sep 30 09:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:24.621200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ef9e3-34ea-49ff-8bd8-75aeeac02c96 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:24.621485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ef9e3-34ea-49ff-8bd8-75aeeac02c96 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:24.621796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ef9e3-34ea-49ff-8bd8-75aeeac02c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:24.694383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:24.697093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fc8ef9e3-34ea-49ff-8bd8-75aeeac02c96 None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:24.698944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fc8ef9e3-34ea-49ff-8bd8-75aeeac02c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:24.821235 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 710/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 30 09:29:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:27.410916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523963d-ecc9-481d-8901-7455d2ecee06 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:27.411351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523963d-ecc9-481d-8901-7455d2ecee06 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:27.411459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523963d-ecc9-481d-8901-7455d2ecee06 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:27.412793 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:27.418045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ce300-6ebd-4cc8-b5e2-36f45a8d5c5e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:27.418497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ce300-6ebd-4cc8-b5e2-36f45a8d5c5e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:27.418611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ce300-6ebd-4cc8-b5e2-36f45a8d5c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:27.486117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f14ce300-6ebd-4cc8-b5e2-36f45a8d5c5e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:27.487639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f14ce300-6ebd-4cc8-b5e2-36f45a8d5c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:27.548554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:27.590610 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:27.881308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523963d-ecc9-481d-8901-7455d2ecee06 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:27.882900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c523963d-ecc9-481d-8901-7455d2ecee06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:28.027498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523963d-ecc9-481d-8901-7455d2ecee06 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:28.099492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:28.140335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73624fe2-28d0-4d17-b6f2-6f3a1fcb4ffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:28.140949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624fe2-28d0-4d17-b6f2-6f3a1fcb4ffb service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:28.141312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624fe2-28d0-4d17-b6f2-6f3a1fcb4ffb service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:28.141620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624fe2-28d0-4d17-b6f2-6f3a1fcb4ffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:28.143300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-73624fe2-28d0-4d17-b6f2-6f3a1fcb4ffb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:28.231221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624fe2-28d0-4d17-b6f2-6f3a1fcb4ffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:28.232832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624fe2-28d0-4d17-b6f2-6f3a1fcb4ffb service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:28.250900 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 705/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:29.412662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495ddc-f2e0-4428-977d-268311e62a07 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:29.413120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495ddc-f2e0-4428-977d-268311e62a07 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:29.413238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495ddc-f2e0-4428-977d-268311e62a07 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:29.414619 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:29.421159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8c4b4-5c8e-4022-93c1-79f0f76c47c1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:29.421599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8c4b4-5c8e-4022-93c1-79f0f76c47c1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:29.421743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8c4b4-5c8e-4022-93c1-79f0f76c47c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:29.496936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d6a8c4b4-5c8e-4022-93c1-79f0f76c47c1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:29.498187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d6a8c4b4-5c8e-4022-93c1-79f0f76c47c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:29.526294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:29.599256 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:29.749171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7495ddc-f2e0-4428-977d-268311e62a07 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:29.750656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e7495ddc-f2e0-4428-977d-268311e62a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:29.893589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7495ddc-f2e0-4428-977d-268311e62a07 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:29.933088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:30.010985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45434d2c-b52b-4509-9ea4-9ad2e8c1bdd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:30.011494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434d2c-b52b-4509-9ea4-9ad2e8c1bdd4 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:30.011771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434d2c-b52b-4509-9ea4-9ad2e8c1bdd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:30.012040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45434d2c-b52b-4509-9ea4-9ad2e8c1bdd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:30.013657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-45434d2c-b52b-4509-9ea4-9ad2e8c1bdd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:30.105509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45434d2c-b52b-4509-9ea4-9ad2e8c1bdd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:30.107132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45434d2c-b52b-4509-9ea4-9ad2e8c1bdd4 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:30.125125 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 713/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:31.320220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14e0a-4cff-4e3a-bb7a-56833ff18e0e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:31.320785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14e0a-4cff-4e3a-bb7a-56833ff18e0e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:31.320785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14e0a-4cff-4e3a-bb7a-56833ff18e0e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:31.322137 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:31.328503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c9dee-29a3-4158-b513-2a2cd50d0d4e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:31.328936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c9dee-29a3-4158-b513-2a2cd50d0d4e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:31.329041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c9dee-29a3-4158-b513-2a2cd50d0d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:31.384085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:31.406938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0c8c9dee-29a3-4158-b513-2a2cd50d0d4e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:31.408726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c8c9dee-29a3-4158-b513-2a2cd50d0d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:31.503279 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:31.655656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14e0a-4cff-4e3a-bb7a-56833ff18e0e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:31.657192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2fb14e0a-4cff-4e3a-bb7a-56833ff18e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:31.802249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14e0a-4cff-4e3a-bb7a-56833ff18e0e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:31.814192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:31.914048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41f7b1-fa3d-4598-89cd-1e922be03492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:31.914581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f7b1-fa3d-4598-89cd-1e922be03492 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:31.914856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f7b1-fa3d-4598-89cd-1e922be03492 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:31.915125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41f7b1-fa3d-4598-89cd-1e922be03492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:31.916733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0b41f7b1-fa3d-4598-89cd-1e922be03492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:32.006284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41f7b1-fa3d-4598-89cd-1e922be03492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:32.007867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41f7b1-fa3d-4598-89cd-1e922be03492 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:32.026267 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 708/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:33.139201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da763b9d-f43f-4f43-a7b2-0187628f1737 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:33.139751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da763b9d-f43f-4f43-a7b2-0187628f1737 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:33.139751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da763b9d-f43f-4f43-a7b2-0187628f1737 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:33.141072 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:33.148392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4362c-071e-4d05-9f66-5ffdf68cd2d9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:33.148870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4362c-071e-4d05-9f66-5ffdf68cd2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:33.148975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4362c-071e-4d05-9f66-5ffdf68cd2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:33.168494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:33.228403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c8f4362c-071e-4d05-9f66-5ffdf68cd2d9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:33.229608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c8f4362c-071e-4d05-9f66-5ffdf68cd2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:33.325592 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:33.479381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da763b9d-f43f-4f43-a7b2-0187628f1737 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:33.480903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-da763b9d-f43f-4f43-a7b2-0187628f1737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:33.595509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:33.632114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da763b9d-f43f-4f43-a7b2-0187628f1737 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:33.744085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e075be-cc50-4033-959b-3d37be12bd69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:33.744612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e075be-cc50-4033-959b-3d37be12bd69 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:33.744898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e075be-cc50-4033-959b-3d37be12bd69 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:33.745169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e075be-cc50-4033-959b-3d37be12bd69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:33.746848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-05e075be-cc50-4033-959b-3d37be12bd69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:33.821909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:33.835710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e075be-cc50-4033-959b-3d37be12bd69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:33.837311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e075be-cc50-4033-959b-3d37be12bd69 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:33.855480 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 716/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:34.969363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67620907-9a39-4119-b0a8-20a4bde62f6f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:34.969944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67620907-9a39-4119-b0a8-20a4bde62f6f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:34.969944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67620907-9a39-4119-b0a8-20a4bde62f6f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:34.971525 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:34.977606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56eba-dfb2-4c00-9503-161ab3772d46 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:34.978099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56eba-dfb2-4c00-9503-161ab3772d46 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:34.978198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae56eba-dfb2-4c00-9503-161ab3772d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:35.054987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6ae56eba-dfb2-4c00-9503-161ab3772d46 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:35.056188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6ae56eba-dfb2-4c00-9503-161ab3772d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:35.149975 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:35.301115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67620907-9a39-4119-b0a8-20a4bde62f6f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:35.302590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-67620907-9a39-4119-b0a8-20a4bde62f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:35.384219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:35.447711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67620907-9a39-4119-b0a8-20a4bde62f6f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:35.559771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39afe98-d2d9-4bfa-a350-93faed0dc4bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:35.560324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39afe98-d2d9-4bfa-a350-93faed0dc4bc service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:35.560622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39afe98-d2d9-4bfa-a350-93faed0dc4bc service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:35.560892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39afe98-d2d9-4bfa-a350-93faed0dc4bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:35.562557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a39afe98-d2d9-4bfa-a350-93faed0dc4bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:35.612040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:35.652643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39afe98-d2d9-4bfa-a350-93faed0dc4bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:35.654279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39afe98-d2d9-4bfa-a350-93faed0dc4bc service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:35.671844 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 711/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:36.723849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005ba1-0023-4156-a426-ca938ecd1921 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:36.724419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005ba1-0023-4156-a426-ca938ecd1921 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:36.724419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11005ba1-0023-4156-a426-ca938ecd1921 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:36.725669 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:36.732026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae32bc-1aae-4305-b728-430d3195cef3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:36.732503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae32bc-1aae-4305-b728-430d3195cef3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:36.732613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae32bc-1aae-4305-b728-430d3195cef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:36.809066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-78ae32bc-1aae-4305-b728-430d3195cef3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:36.810308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-78ae32bc-1aae-4305-b728-430d3195cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:36.903694 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:37.055364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11005ba1-0023-4156-a426-ca938ecd1921 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:37.056855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-11005ba1-0023-4156-a426-ca938ecd1921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:37.105294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:37.201467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11005ba1-0023-4156-a426-ca938ecd1921 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:37.311130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33fa02-6da0-4c58-bafa-ef39d8e401cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:37.311654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33fa02-6da0-4c58-bafa-ef39d8e401cb service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:37.311929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33fa02-6da0-4c58-bafa-ef39d8e401cb service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:37.312195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33fa02-6da0-4c58-bafa-ef39d8e401cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:37.313800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7a33fa02-6da0-4c58-bafa-ef39d8e401cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:37.321216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:37.407683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33fa02-6da0-4c58-bafa-ef39d8e401cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:37.408769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33fa02-6da0-4c58-bafa-ef39d8e401cb service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:37.426917 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 719/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:38.487701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbc8bb-6814-43c3-a030-dfdb0dee53b0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:38.488265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbc8bb-6814-43c3-a030-dfdb0dee53b0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:38.488265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbc8bb-6814-43c3-a030-dfdb0dee53b0 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:38.489599 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:38.495990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3686d6-dee8-452b-8788-fc3d8921ae79 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:38.496394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3686d6-dee8-452b-8788-fc3d8921ae79 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:38.496541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3686d6-dee8-452b-8788-fc3d8921ae79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:38.573250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ba3686d6-dee8-452b-8788-fc3d8921ae79 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:38.574500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ba3686d6-dee8-452b-8788-fc3d8921ae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:38.641491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:38.669620 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:38.820993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbc8bb-6814-43c3-a030-dfdb0dee53b0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:38.822548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1bfbc8bb-6814-43c3-a030-dfdb0dee53b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:38.840093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:38.966388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbc8bb-6814-43c3-a030-dfdb0dee53b0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:39.055310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:39.080471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e464918-400a-44e1-b30b-a6e5b2a2e3a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:39.081080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e464918-400a-44e1-b30b-a6e5b2a2e3a7 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:39.081364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e464918-400a-44e1-b30b-a6e5b2a2e3a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:39.081620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e464918-400a-44e1-b30b-a6e5b2a2e3a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:39.083264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e464918-400a-44e1-b30b-a6e5b2a2e3a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:39.177394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e464918-400a-44e1-b30b-a6e5b2a2e3a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:39.179030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e464918-400a-44e1-b30b-a6e5b2a2e3a7 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:39.196421 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 714/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:40.243556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73977-21ff-434d-9c8d-29685fea4615 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:40.244128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73977-21ff-434d-9c8d-29685fea4615 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:40.244128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea73977-21ff-434d-9c8d-29685fea4615 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:40.245481 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:40.251983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d804b-a490-45de-9f6b-7124adcefe34 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:40.252402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d804b-a490-45de-9f6b-7124adcefe34 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:40.252535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d804b-a490-45de-9f6b-7124adcefe34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:40.327738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-db2d804b-a490-45de-9f6b-7124adcefe34 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:40.328939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-db2d804b-a490-45de-9f6b-7124adcefe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:40.370307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:40.432150 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:40.587720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea73977-21ff-434d-9c8d-29685fea4615 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:40.589269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ea73977-21ff-434d-9c8d-29685fea4615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:40.736740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea73977-21ff-434d-9c8d-29685fea4615 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:40.793345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:40.851406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760c7f-2d82-4354-ae83-0c45be554011 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:40.851915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760c7f-2d82-4354-ae83-0c45be554011 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:40.852190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760c7f-2d82-4354-ae83-0c45be554011 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:40.852457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760c7f-2d82-4354-ae83-0c45be554011 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:40.854076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5e760c7f-2d82-4354-ae83-0c45be554011 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:40.944606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e760c7f-2d82-4354-ae83-0c45be554011 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:40.946236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e760c7f-2d82-4354-ae83-0c45be554011 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:40.963708 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 722/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:42.006901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9c6f9-d394-4b3e-a166-df606797010b None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:42.007473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9c6f9-d394-4b3e-a166-df606797010b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:42.007473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9c6f9-d394-4b3e-a166-df606797010b None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:42.008781 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:42.014929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f039ee9-2e59-4de0-93a0-257aba4be660 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:42.015376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f039ee9-2e59-4de0-93a0-257aba4be660 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:42.015490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f039ee9-2e59-4de0-93a0-257aba4be660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:42.093519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7f039ee9-2e59-4de0-93a0-257aba4be660 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:42.094765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7f039ee9-2e59-4de0-93a0-257aba4be660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:42.097329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:42.191675 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:42.340978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9c6f9-d394-4b3e-a166-df606797010b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:42.342467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-38f9c6f9-d394-4b3e-a166-df606797010b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:42.489467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9c6f9-d394-4b3e-a166-df606797010b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:42.517138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:42.720019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfde8e8-7e7a-483a-a826-746b33ed7cef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:42.721147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde8e8-7e7a-483a-a826-746b33ed7cef service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:42.721418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde8e8-7e7a-483a-a826-746b33ed7cef service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:42.721710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde8e8-7e7a-483a-a826-746b33ed7cef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:42.723300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ccfde8e8-7e7a-483a-a826-746b33ed7cef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:42.809140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfde8e8-7e7a-483a-a826-746b33ed7cef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:42.810734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfde8e8-7e7a-483a-a826-746b33ed7cef service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:42.829321 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 717/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:44.918994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18359b7-396f-4b44-88c7-99309bcf1957 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:44.919560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18359b7-396f-4b44-88c7-99309bcf1957 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:44.919560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18359b7-396f-4b44-88c7-99309bcf1957 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:44.920904 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:44.926270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c71fd-9018-49a9-aee1-c448256ee6c7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:44.926696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c71fd-9018-49a9-aee1-c448256ee6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:44.926816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c71fd-9018-49a9-aee1-c448256ee6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:44.967593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:44.992629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-255c71fd-9018-49a9-aee1-c448256ee6c7 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:44.993867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-255c71fd-9018-49a9-aee1-c448256ee6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:45.089092 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:45.380105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18359b7-396f-4b44-88c7-99309bcf1957 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:45.381706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d18359b7-396f-4b44-88c7-99309bcf1957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:45.484641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18359b7-396f-4b44-88c7-99309bcf1957 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:45.516717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:45.557961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc03cee-8a2a-41a6-8a21-1e5c10f07c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:45.558557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc03cee-8a2a-41a6-8a21-1e5c10f07c9f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:45.558829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc03cee-8a2a-41a6-8a21-1e5c10f07c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:45.559095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc03cee-8a2a-41a6-8a21-1e5c10f07c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:45.560664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bdc03cee-8a2a-41a6-8a21-1e5c10f07c9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:45.667348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc03cee-8a2a-41a6-8a21-1e5c10f07c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:45.668699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc03cee-8a2a-41a6-8a21-1e5c10f07c9f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:45.686017 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 725/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:47.962637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68c882-d468-4a84-9f75-6269c1f43cb2 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:47.963207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68c882-d468-4a84-9f75-6269c1f43cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:47.963207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68c882-d468-4a84-9f75-6269c1f43cb2 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:47.964581 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:47.969879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3028b5e-3371-4bf2-9724-3f37176d0d14 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:47.970957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3028b5e-3371-4bf2-9724-3f37176d0d14 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:47.970957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3028b5e-3371-4bf2-9724-3f37176d0d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:48.036930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c3028b5e-3371-4bf2-9724-3f37176d0d14 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:48.038170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c3028b5e-3371-4bf2-9724-3f37176d0d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:48.051588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:48.137952 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:48.424510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68c882-d468-4a84-9f75-6269c1f43cb2 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:48.426081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4a68c882-d468-4a84-9f75-6269c1f43cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:48.529127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68c882-d468-4a84-9f75-6269c1f43cb2 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:48.542391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:48.616214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8836e4-73ab-48db-97e0-84b1e74097bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:48.616813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8836e4-73ab-48db-97e0-84b1e74097bb service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:48.617086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8836e4-73ab-48db-97e0-84b1e74097bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:48.617352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8836e4-73ab-48db-97e0-84b1e74097bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:48.618968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0e8836e4-73ab-48db-97e0-84b1e74097bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:48.725714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8836e4-73ab-48db-97e0-84b1e74097bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:48.727042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8836e4-73ab-48db-97e0-84b1e74097bb service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:48.744509 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 720/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:51.231605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b46b3-c5e7-48fa-b646-9411e1712347 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:51.232165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b46b3-c5e7-48fa-b646-9411e1712347 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:51.232165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b46b3-c5e7-48fa-b646-9411e1712347 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:51.233487 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:51.240125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec7b2a-6a07-416e-bc9d-87111069de3f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:51.240607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec7b2a-6a07-416e-bc9d-87111069de3f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:51.240711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec7b2a-6a07-416e-bc9d-87111069de3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:51.291288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:51.310985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3dec7b2a-6a07-416e-bc9d-87111069de3f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:51.312188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3dec7b2a-6a07-416e-bc9d-87111069de3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:51.411776 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:51.701293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b46b3-c5e7-48fa-b646-9411e1712347 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:51.702883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-796b46b3-c5e7-48fa-b646-9411e1712347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:51.810497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b46b3-c5e7-48fa-b646-9411e1712347 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:51.850568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:51.885607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a34156-617e-4bb6-b1c0-22e96ab08218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:51.886237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a34156-617e-4bb6-b1c0-22e96ab08218 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:51.886517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a34156-617e-4bb6-b1c0-22e96ab08218 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:51.886790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a34156-617e-4bb6-b1c0-22e96ab08218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:51.888382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-81a34156-617e-4bb6-b1c0-22e96ab08218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:51.999046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a34156-617e-4bb6-b1c0-22e96ab08218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:52.000399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a34156-617e-4bb6-b1c0-22e96ab08218 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:52.019558 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 728/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:54.318583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923531c-2710-405e-8ebd-53a4c49d360b None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:54.319146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923531c-2710-405e-8ebd-53a4c49d360b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:54.319146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923531c-2710-405e-8ebd-53a4c49d360b None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:54.320465 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:54.325723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9c52d-e39c-40f4-b03f-fed10bb62ca8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:54.326124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9c52d-e39c-40f4-b03f-fed10bb62ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:54.326272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9c52d-e39c-40f4-b03f-fed10bb62ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:54.391492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ffa9c52d-e39c-40f4-b03f-fed10bb62ca8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:54.393606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ffa9c52d-e39c-40f4-b03f-fed10bb62ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:54.416602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:54.493679 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:54.789421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923531c-2710-405e-8ebd-53a4c49d360b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:54.790980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7923531c-2710-405e-8ebd-53a4c49d360b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:54.891820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923531c-2710-405e-8ebd-53a4c49d360b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:54.912420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:54.965418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee626582-3c91-4f8c-894c-3afb681b3a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:54.966041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee626582-3c91-4f8c-894c-3afb681b3a40 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:54.966315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee626582-3c91-4f8c-894c-3afb681b3a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:54.966581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee626582-3c91-4f8c-894c-3afb681b3a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:54.968165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ee626582-3c91-4f8c-894c-3afb681b3a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:55.080203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee626582-3c91-4f8c-894c-3afb681b3a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:55.081435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee626582-3c91-4f8c-894c-3afb681b3a40 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:55.098695 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 723/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:29:57.321378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8360-e28c-4151-83b8-b67fdba261e7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:57.321986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8360-e28c-4151-83b8-b67fdba261e7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:57.321986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad8360-e28c-4151-83b8-b67fdba261e7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:57.323308 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:29:57.328723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ea34-c4af-4c5d-8858-a8e1e68f74dd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:57.329146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ea34-c4af-4c5d-8858-a8e1e68f74dd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:57.329872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ea34-c4af-4c5d-8858-a8e1e68f74dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:57.394992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:57.395880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d059ea34-c4af-4c5d-8858-a8e1e68f74dd None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:29:57.398230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d059ea34-c4af-4c5d-8858-a8e1e68f74dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:57.492630 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:29:57.775043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad8360-e28c-4151-83b8-b67fdba261e7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:29:57.776579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-67ad8360-e28c-4151-83b8-b67fdba261e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:57.876710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad8360-e28c-4151-83b8-b67fdba261e7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:29:57.924602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:29:57.949117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac545f2a-e448-43dc-b59d-9e8b5b96e7e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:29:57.949741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545f2a-e448-43dc-b59d-9e8b5b96e7e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:29:57.950018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545f2a-e448-43dc-b59d-9e8b5b96e7e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:29:57.950283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac545f2a-e448-43dc-b59d-9e8b5b96e7e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:29:57.951868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ac545f2a-e448-43dc-b59d-9e8b5b96e7e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:29:58.058922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac545f2a-e448-43dc-b59d-9e8b5b96e7e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:29:58.060285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac545f2a-e448-43dc-b59d-9e8b5b96e7e5 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:29:58.078482 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 731/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:00.276660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bf5b-2b53-4327-bb90-b14b9985ee03 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:00.277229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bf5b-2b53-4327-bb90-b14b9985ee03 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:00.277229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bf5b-2b53-4327-bb90-b14b9985ee03 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:00.278555 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:00.284398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5881-c4d0-40b5-acf7-fe5d13fcc251 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:00.284836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5881-c4d0-40b5-acf7-fe5d13fcc251 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:00.284950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5881-c4d0-40b5-acf7-fe5d13fcc251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:00.350758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-af9a5881-c4d0-40b5-acf7-fe5d13fcc251 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:00.351939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-af9a5881-c4d0-40b5-acf7-fe5d13fcc251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:00.383738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:00.453502 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:00.738661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494bf5b-2b53-4327-bb90-b14b9985ee03 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:00.740189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b494bf5b-2b53-4327-bb90-b14b9985ee03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:00.840066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494bf5b-2b53-4327-bb90-b14b9985ee03 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:00.871376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:00.916832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63877e-6c6c-4905-830b-24f0ce09d54e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:00.917433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63877e-6c6c-4905-830b-24f0ce09d54e service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:00.917741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63877e-6c6c-4905-830b-24f0ce09d54e service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:00.918010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63877e-6c6c-4905-830b-24f0ce09d54e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:00.919592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3f63877e-6c6c-4905-830b-24f0ce09d54e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:01.026026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63877e-6c6c-4905-830b-24f0ce09d54e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:01.027472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63877e-6c6c-4905-830b-24f0ce09d54e service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:01.046452 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 726/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:03.280827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fde6e-39c9-4cd2-91d7-7e882f361697 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:03.281406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fde6e-39c9-4cd2-91d7-7e882f361697 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:03.281406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fde6e-39c9-4cd2-91d7-7e882f361697 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:03.283326 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:03.288741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d47069-394c-4a13-af08-1dc6e97ef2e5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:03.289161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d47069-394c-4a13-af08-1dc6e97ef2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:03.289314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d47069-394c-4a13-af08-1dc6e97ef2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:03.354964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-27d47069-394c-4a13-af08-1dc6e97ef2e5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:03.356173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-27d47069-394c-4a13-af08-1dc6e97ef2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:03.364882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:03.456754 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:03.742857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fde6e-39c9-4cd2-91d7-7e882f361697 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:03.744418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-495fde6e-39c9-4cd2-91d7-7e882f361697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:03.847850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fde6e-39c9-4cd2-91d7-7e882f361697 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:03.902252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:03.920332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff115532-b623-401b-8a5b-15c9600a705f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:03.920930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff115532-b623-401b-8a5b-15c9600a705f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:03.921206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff115532-b623-401b-8a5b-15c9600a705f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:03.921475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff115532-b623-401b-8a5b-15c9600a705f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:03.923686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ff115532-b623-401b-8a5b-15c9600a705f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:04.030678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff115532-b623-401b-8a5b-15c9600a705f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:04.032043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff115532-b623-401b-8a5b-15c9600a705f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:04.052524 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 734/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:06.235720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b14cd7-a2f2-4555-8e91-a102337b31db None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:06.236297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b14cd7-a2f2-4555-8e91-a102337b31db None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:06.236297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b14cd7-a2f2-4555-8e91-a102337b31db None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:06.237593 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:06.243525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5472df6-8d98-47cf-a600-2b712f6f03a9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:06.243940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5472df6-8d98-47cf-a600-2b712f6f03a9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:06.244070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5472df6-8d98-47cf-a600-2b712f6f03a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:06.310232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e5472df6-8d98-47cf-a600-2b712f6f03a9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:06.311407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e5472df6-8d98-47cf-a600-2b712f6f03a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:06.349159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:06.407994 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:06.699326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b14cd7-a2f2-4555-8e91-a102337b31db None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:06.700865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c3b14cd7-a2f2-4555-8e91-a102337b31db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:06.803003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b14cd7-a2f2-4555-8e91-a102337b31db None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:06.840090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:06.882612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e101b-312e-4443-b699-fb50d380407d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:06.883224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e101b-312e-4443-b699-fb50d380407d service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:06.883504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e101b-312e-4443-b699-fb50d380407d service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:06.883777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e101b-312e-4443-b699-fb50d380407d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:06.885398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-db6e101b-312e-4443-b699-fb50d380407d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:06.994237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e101b-312e-4443-b699-fb50d380407d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:06.995657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e101b-312e-4443-b699-fb50d380407d service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:07.013358 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 729/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:09.181204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99104-f040-48ac-8d4e-66e3666299d7 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:09.181825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99104-f040-48ac-8d4e-66e3666299d7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:09.181825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99104-f040-48ac-8d4e-66e3666299d7 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:09.183118 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:09.188501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40552ffb-1d0e-43c3-84e7-35eacb759f3f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:09.188951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40552ffb-1d0e-43c3-84e7-35eacb759f3f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:09.189065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40552ffb-1d0e-43c3-84e7-35eacb759f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:09.254621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-40552ffb-1d0e-43c3-84e7-35eacb759f3f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:09.255839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-40552ffb-1d0e-43c3-84e7-35eacb759f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:09.274122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:09.352022 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:09.634322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b99104-f040-48ac-8d4e-66e3666299d7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:09.635897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c1b99104-f040-48ac-8d4e-66e3666299d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:09.738354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b99104-f040-48ac-8d4e-66e3666299d7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:09.804649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:09.811720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d852e5f-19a6-44d0-8a02-810ad6b80f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:09.812327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d852e5f-19a6-44d0-8a02-810ad6b80f1d service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:09.812607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d852e5f-19a6-44d0-8a02-810ad6b80f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:09.812880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d852e5f-19a6-44d0-8a02-810ad6b80f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:09.815154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1d852e5f-19a6-44d0-8a02-810ad6b80f1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:09.921085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d852e5f-19a6-44d0-8a02-810ad6b80f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:09.922481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d852e5f-19a6-44d0-8a02-810ad6b80f1d service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:09.940889 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 737/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:12.101185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55d474-9d45-4810-a2c0-4d7596d2ab06 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:12.101786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55d474-9d45-4810-a2c0-4d7596d2ab06 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:12.101786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55d474-9d45-4810-a2c0-4d7596d2ab06 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:12.103077 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:12.108313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206ffc-60d2-463b-b58b-77807e6877b1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:12.108752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206ffc-60d2-463b-b58b-77807e6877b1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:12.108857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71206ffc-60d2-463b-b58b-77807e6877b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:12.174234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-71206ffc-60d2-463b-b58b-77807e6877b1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:12.175427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-71206ffc-60d2-463b-b58b-77807e6877b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:12.223621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:12.274363 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:12.557096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55d474-9d45-4810-a2c0-4d7596d2ab06 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:12.558705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e55d474-9d45-4810-a2c0-4d7596d2ab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:12.660514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55d474-9d45-4810-a2c0-4d7596d2ab06 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:12.706790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:12.731860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16773c-e4e2-4b5e-883b-638b6adab099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:12.732463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16773c-e4e2-4b5e-883b-638b6adab099 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:12.732737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16773c-e4e2-4b5e-883b-638b6adab099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:12.733002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16773c-e4e2-4b5e-883b-638b6adab099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:12.734601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2e16773c-e4e2-4b5e-883b-638b6adab099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:12.841314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16773c-e4e2-4b5e-883b-638b6adab099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:12.842775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16773c-e4e2-4b5e-883b-638b6adab099 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:12.868367 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 732/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:22.272502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=22 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:22.866153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=66 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:33.106906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fce03b-82db-4751-a947-e50f74746d3a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:33.107469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fce03b-82db-4751-a947-e50f74746d3a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:33.107469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fce03b-82db-4751-a947-e50f74746d3a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:33.108897 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 739/1471] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:33.114552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab2f7d-8bb4-4d46-9456-5a7d26ec4613 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:33.114964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab2f7d-8bb4-4d46-9456-5a7d26ec4613 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:33.115100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab2f7d-8bb4-4d46-9456-5a7d26ec4613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:33.181731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-79ab2f7d-8bb4-4d46-9456-5a7d26ec4613 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:33.182936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-79ab2f7d-8bb4-4d46-9456-5a7d26ec4613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:33.280875 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 733/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:33.587661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1cff2-a38b-451e-839e-3880124b94d0 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:33.587952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1cff2-a38b-451e-839e-3880124b94d0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:33.588247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1cff2-a38b-451e-839e-3880124b94d0 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:33.590434 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 740/1473] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 30 09:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:33.595638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca18dec-3a09-4e55-a101-467dd321d6de None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:33.595922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca18dec-3a09-4e55-a101-467dd321d6de None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:33.596201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca18dec-3a09-4e55-a101-467dd321d6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:33.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fca18dec-3a09-4e55-a101-467dd321d6de None None] MFA Rules not processed for user `7fe7a090bd7941bb827d301aa4949857`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:33.673736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fca18dec-3a09-4e55-a101-467dd321d6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:33.695670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:33.755087 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 734/1474] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 30 09:30:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:33.762719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1cff2-a38b-451e-839e-3880124b94d0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:33.764228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fee1cff2-a38b-451e-839e-3880124b94d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:33.867747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1cff2-a38b-451e-839e-3880124b94d0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:33.942060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdd61e-6c87-4abc-9fa6-55a7939a2747 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fe7a090bd7941bb827d301aa4949857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:33.942593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd61e-6c87-4abc-9fa6-55a7939a2747 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:33.942873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd61e-6c87-4abc-9fa6-55a7939a2747 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:33.943143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdd61e-6c87-4abc-9fa6-55a7939a2747 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:33.944739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bcfdd61e-6c87-4abc-9fa6-55a7939a2747 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:33.998218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:34.054616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdd61e-6c87-4abc-9fa6-55a7939a2747 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:34.056070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdd61e-6c87-4abc-9fa6-55a7939a2747 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:34.074442 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 741/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:36.381908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87edc5d-cb4b-463a-9e74-671a81f753e4 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:36.382473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87edc5d-cb4b-463a-9e74-671a81f753e4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:36.382573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87edc5d-cb4b-463a-9e74-671a81f753e4 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:36.384515 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 735/1476] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:30:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:36.389878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f5228-b1b6-444f-bf16-c08a5995cf27 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:36.390300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f5228-b1b6-444f-bf16-c08a5995cf27 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:36.390680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f5228-b1b6-444f-bf16-c08a5995cf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:36.456734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5e9f5228-b1b6-444f-bf16-c08a5995cf27 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:36.457961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5e9f5228-b1b6-444f-bf16-c08a5995cf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:36.555020 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 742/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:36.839317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87edc5d-cb4b-463a-9e74-671a81f753e4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:36.840757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b87edc5d-cb4b-463a-9e74-671a81f753e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:36.972507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87edc5d-cb4b-463a-9e74-671a81f753e4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:37.022757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:37.067261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e76d5-0421-4100-9c0f-eb54c7aa9a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:37.067874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e76d5-0421-4100-9c0f-eb54c7aa9a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:37.068157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e76d5-0421-4100-9c0f-eb54c7aa9a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:37.068430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e76d5-0421-4100-9c0f-eb54c7aa9a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:37.069070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e76d5-0421-4100-9c0f-eb54c7aa9a39 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:37.070869 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-f45e76d5-0421-4100-9c0f-eb54c7aa9a39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:30:37.070869 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Sep 30 09:30:37.070869 dv-cinder-master-3-smb devstack@keystone.service[81372]: Sep 30 09:30:37.072017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e76d5-0421-4100-9c0f-eb54c7aa9a39 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:37.091344 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 736/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 30 09:30:36 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:30:39.277037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ca438-4b74-4263-a788-e2769aebf217 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:39.277680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ca438-4b74-4263-a788-e2769aebf217 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:39.277680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ca438-4b74-4263-a788-e2769aebf217 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:39.278964 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 743/1479] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:39.284806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad9cf2-b395-431d-8fad-a4e9f75f09fb None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:39.285247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad9cf2-b395-431d-8fad-a4e9f75f09fb None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:39.285361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad9cf2-b395-431d-8fad-a4e9f75f09fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:39.351530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-25ad9cf2-b395-431d-8fad-a4e9f75f09fb None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:39.352737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-25ad9cf2-b395-431d-8fad-a4e9f75f09fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:39.448980 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 737/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:39.747625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ca438-4b74-4263-a788-e2769aebf217 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:39.749222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-726ca438-4b74-4263-a788-e2769aebf217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:39.760813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:39.909358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ca438-4b74-4263-a788-e2769aebf217 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:39.997741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:40.032258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f94ca9-930f-4426-8f4f-8fa9c88f4c1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:40.032860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94ca9-930f-4426-8f4f-8fa9c88f4c1d service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:40.033134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94ca9-930f-4426-8f4f-8fa9c88f4c1d service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:40.033399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94ca9-930f-4426-8f4f-8fa9c88f4c1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:40.035025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-26f94ca9-930f-4426-8f4f-8fa9c88f4c1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:40.129341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f94ca9-930f-4426-8f4f-8fa9c88f4c1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:40.131014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f94ca9-930f-4426-8f4f-8fa9c88f4c1d service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:40.150333 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 744/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:42.520936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bce7db-f7a5-471c-a883-e647f74d3157 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:42.521485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bce7db-f7a5-471c-a883-e647f74d3157 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:42.521485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bce7db-f7a5-471c-a883-e647f74d3157 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:42.523196 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 738/1482] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:42.530596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ecb2-5b25-4bc0-99b7-46fb5043ce80 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:42.531097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ecb2-5b25-4bc0-99b7-46fb5043ce80 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:42.531648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ecb2-5b25-4bc0-99b7-46fb5043ce80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:42.604237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f652ecb2-5b25-4bc0-99b7-46fb5043ce80 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:42.605700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f652ecb2-5b25-4bc0-99b7-46fb5043ce80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:42.657163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:42.710002 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 745/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:43.012439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bce7db-f7a5-471c-a883-e647f74d3157 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:43.014093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-01bce7db-f7a5-471c-a883-e647f74d3157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:43.040746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:43.127674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bce7db-f7a5-471c-a883-e647f74d3157 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:43.205488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e789b50-d214-426c-b079-4becfff2cb36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:43.206150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e789b50-d214-426c-b079-4becfff2cb36 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:43.206429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e789b50-d214-426c-b079-4becfff2cb36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:43.206702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e789b50-d214-426c-b079-4becfff2cb36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:43.208322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6e789b50-d214-426c-b079-4becfff2cb36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:43.289170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:43.325002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e789b50-d214-426c-b079-4becfff2cb36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:43.326474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e789b50-d214-426c-b079-4becfff2cb36 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:43.345568 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 739/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:45.650526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f1cc-1cb6-47d1-a25b-6f4145ae626e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:45.651066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f1cc-1cb6-47d1-a25b-6f4145ae626e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:45.651066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f1cc-1cb6-47d1-a25b-6f4145ae626e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:45.652413 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 746/1485] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:45.658493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5febd3-e70f-4c71-90fe-8925d62cf7f1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:45.658941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5febd3-e70f-4c71-90fe-8925d62cf7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:45.659061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5febd3-e70f-4c71-90fe-8925d62cf7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:45.729215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0e5febd3-e70f-4c71-90fe-8925d62cf7f1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:45.730452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0e5febd3-e70f-4c71-90fe-8925d62cf7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:45.830849 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 740/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:46.121067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05f1cc-1cb6-47d1-a25b-6f4145ae626e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:46.122624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dd05f1cc-1cb6-47d1-a25b-6f4145ae626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:46.232620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05f1cc-1cb6-47d1-a25b-6f4145ae626e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:46.306522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:46.308198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8848c9-4a63-454d-8b50-9d4973e96f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:46.308878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8848c9-4a63-454d-8b50-9d4973e96f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:46.309393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8848c9-4a63-454d-8b50-9d4973e96f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:46.309767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8848c9-4a63-454d-8b50-9d4973e96f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:46.311881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ad8848c9-4a63-454d-8b50-9d4973e96f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:46.435326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8848c9-4a63-454d-8b50-9d4973e96f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:46.436913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8848c9-4a63-454d-8b50-9d4973e96f1b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:46.455332 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 747/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:48.828364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072376c-d317-4117-9a17-03850124c16c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:48.828935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072376c-d317-4117-9a17-03850124c16c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:48.828935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072376c-d317-4117-9a17-03850124c16c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:48.830244 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 741/1488] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:48.835629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2f288-f120-4d87-b0ff-3ecc5a8f3f26 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:48.836101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2f288-f120-4d87-b0ff-3ecc5a8f3f26 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:48.836222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2f288-f120-4d87-b0ff-3ecc5a8f3f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:48.903689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7da2f288-f120-4d87-b0ff-3ecc5a8f3f26 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:48.904896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7da2f288-f120-4d87-b0ff-3ecc5a8f3f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:48.959377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:49.005222 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 748/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:49.301486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072376c-d317-4117-9a17-03850124c16c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:49.303089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e072376c-d317-4117-9a17-03850124c16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:49.340160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:49.408641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072376c-d317-4117-9a17-03850124c16c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:49.481867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eee563-2ad1-457e-bfd5-a16c1d98bafe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:49.482524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee563-2ad1-457e-bfd5-a16c1d98bafe service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:49.482802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee563-2ad1-457e-bfd5-a16c1d98bafe service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:49.483095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee563-2ad1-457e-bfd5-a16c1d98bafe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:49.484719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-23eee563-2ad1-457e-bfd5-a16c1d98bafe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:49.564651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:49.594444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eee563-2ad1-457e-bfd5-a16c1d98bafe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:49.595863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eee563-2ad1-457e-bfd5-a16c1d98bafe service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:49.614076 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 742/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:30:52.012531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f8362-ec85-45ec-867a-3c7fab2a77e9 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:52.013109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f8362-ec85-45ec-867a-3c7fab2a77e9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:52.013109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f8362-ec85-45ec-867a-3c7fab2a77e9 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:52.014441 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 749/1491] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 30 09:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:30:52.020657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce11bad-b645-482a-bdad-2745dca5b7cc None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:52.021093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce11bad-b645-482a-bdad-2745dca5b7cc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:52.021212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce11bad-b645-482a-bdad-2745dca5b7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:52.088781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8ce11bad-b645-482a-bdad-2745dca5b7cc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:30:52.090025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8ce11bad-b645-482a-bdad-2745dca5b7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:52.187062 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 743/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 30 09:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:30:52.481276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f8362-ec85-45ec-867a-3c7fab2a77e9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:30:52.482835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-010f8362-ec85-45ec-867a-3c7fab2a77e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:52.585471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f8362-ec85-45ec-867a-3c7fab2a77e9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:30:52.659494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92365a-1fda-49ad-9303-d7b8abba5575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:30:52.660094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92365a-1fda-49ad-9303-d7b8abba5575 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:30:52.660376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92365a-1fda-49ad-9303-d7b8abba5575 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:30:52.660647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92365a-1fda-49ad-9303-d7b8abba5575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:30:52.662275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0f92365a-1fda-49ad-9303-d7b8abba5575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:30:52.671448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:30:52.770985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92365a-1fda-49ad-9303-d7b8abba5575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:30:52.772710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92365a-1fda-49ad-9303-d7b8abba5575 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:30:52.790865 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 750/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:31:02.185541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=82 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:31:02.788700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=50 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:12.828605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fff65-81d4-4c86-9fd5-4ecc2e2f5a21 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:12.829207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fff65-81d4-4c86-9fd5-4ecc2e2f5a21 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:12.829207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fff65-81d4-4c86-9fd5-4ecc2e2f5a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:12.904336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f92fff65-81d4-4c86-9fd5-4ecc2e2f5a21 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:32:12.905551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f92fff65-81d4-4c86-9fd5-4ecc2e2f5a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:12.999400 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 744/1494] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:32:13.007473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1dd6b-1606-4524-814e-1e1759c6a8f6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:13.007975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1dd6b-1606-4524-814e-1e1759c6a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:13.008086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1dd6b-1606-4524-814e-1e1759c6a8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:13.086290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-edd1dd6b-1606-4524-814e-1e1759c6a8f6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:32:13.087506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-edd1dd6b-1606-4524-814e-1e1759c6a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:13.186479 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 751/1495] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:32:13.191142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fff65-81d4-4c86-9fd5-4ecc2e2f5a21 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:13.192373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f92fff65-81d4-4c86-9fd5-4ecc2e2f5a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:13.203562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fff65-81d4-4c86-9fd5-4ecc2e2f5a21 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:13.264173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:13.299151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:13.299753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:13.300028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:13.300297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:13.300929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:32:13.301226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:13.302267 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:32:13.302267 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Sep 30 09:32:13.302267 dv-cinder-master-3-smb devstack@keystone.service[81372]: Sep 30 09:32:13.303146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:13.314583 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 745/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:32:13.322793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1dd6b-1606-4524-814e-1e1759c6a8f6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:13.331991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1dd6b-1606-4524-814e-1e1759c6a8f6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:13.394134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:13.428432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:13.429035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:13.429314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:13.429585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:13.430274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:13.432009 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:32:13.432009 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Sep 30 09:32:13.432009 dv-cinder-master-3-smb devstack@keystone.service[81371]: Sep 30 09:32:13.433210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:13.488253 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 752/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:32:13 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:32:13.492450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:13.493984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:13.527415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d4ad5-6c7e-4e63-b0fe-ea73879587cb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:13.619280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:13.619873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:13.620146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:13.620411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:13.621020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:13.622043 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:32:13.622043 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Sep 30 09:32:13.622043 dv-cinder-master-3-smb devstack@keystone.service[81372]: Sep 30 09:32:13.624056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:13.665144 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 746/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:32:13 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:32:13.669293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:13.670976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:13.735836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49c624-e5e6-4db4-a562-d8c94271a624 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:13.819458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:13.830952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:13.831551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:13.831828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:13.832096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:13.832732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:13.833794 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:32:13.833794 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Sep 30 09:32:13.833794 dv-cinder-master-3-smb devstack@keystone.service[81371]: Sep 30 09:32:13.834696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:13.847163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 753/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:32:13.851318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:13.852828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:13.864855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba2cc-a2b0-4b28-aff1-5967bbdbb618 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:13.890513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:13.961825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea3843-a820-4ffd-ad56-75eb42cc7e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:13.962431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea3843-a820-4ffd-ad56-75eb42cc7e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:13.962705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea3843-a820-4ffd-ad56-75eb42cc7e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:13.962970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea3843-a820-4ffd-ad56-75eb42cc7e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:13.963604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea3843-a820-4ffd-ad56-75eb42cc7e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:13.964598 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-85ea3843-a820-4ffd-ad56-75eb42cc7e69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:32:13.964598 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Sep 30 09:32:13.964598 dv-cinder-master-3-smb devstack@keystone.service[81372]: Sep 30 09:32:13.965472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea3843-a820-4ffd-ad56-75eb42cc7e69 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:13.977841 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 747/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:32:13.982673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:13.984211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:14.020298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48fe92-c7c7-4caf-a392-d585ad5420fa admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:14.113280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433be3-2d0e-4c10-9a1d-716c6ba4de5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:14.113881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433be3-2d0e-4c10-9a1d-716c6ba4de5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:14.114166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433be3-2d0e-4c10-9a1d-716c6ba4de5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:14.114444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433be3-2d0e-4c10-9a1d-716c6ba4de5b admin admin] PATH_INFO: `/v3/projects/bebe649190e34449981a61e5836e91b0/users/c06518a57bd546bf8cd1018d56604900/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:14.136891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433be3-2d0e-4c10-9a1d-716c6ba4de5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bebe649190e34449981a61e5836e91b0, user_id=c06518a57bd546bf8cd1018d56604900, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:14.137946 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-bf433be3-2d0e-4c10-9a1d-716c6ba4de5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:32:14.137946 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Sep 30 09:32:14.137946 dv-cinder-master-3-smb devstack@keystone.service[81371]: Sep 30 09:32:14.140113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433be3-2d0e-4c10-9a1d-716c6ba4de5b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:14.164430 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 754/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:32:13 2022] PUT /identity/v3/projects/bebe649190e34449981a61e5836e91b0/users/c06518a57bd546bf8cd1018d56604900/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:32:14.176983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0dccf-c4e1-4772-979b-f2379fe11903 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:14.177265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0dccf-c4e1-4772-979b-f2379fe11903 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:14.177537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0dccf-c4e1-4772-979b-f2379fe11903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:14.234973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8eb0dccf-c4e1-4772-979b-f2379fe11903 None None] MFA Rules not processed for user `c06518a57bd546bf8cd1018d56604900`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:32:14.236185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8eb0dccf-c4e1-4772-979b-f2379fe11903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:14.312157 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 748/1502] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:32:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:32:14.325408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694de8f1-0b1a-4f79-a6ce-8634c2c38a51 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:14.325720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694de8f1-0b1a-4f79-a6ce-8634c2c38a51 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:14.326073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694de8f1-0b1a-4f79-a6ce-8634c2c38a51 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:14.327724 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 755/1503] 10.222.0.15 () {54 vars in 963 bytes} [Fri Sep 30 09:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:32:14.332727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644740-2f20-4b51-b755-393c08f71617 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:14.333021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644740-2f20-4b51-b755-393c08f71617 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:14.333303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644740-2f20-4b51-b755-393c08f71617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:14.369482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:14.416123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3b644740-2f20-4b51-b755-393c08f71617 None None] MFA Rules not processed for user `2c1784d615d8439a9a4cb9fea305a41b`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:32:14.417376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3b644740-2f20-4b51-b755-393c08f71617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:14.494286 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 749/1504] 10.222.0.15 () {58 vars in 1060 bytes} [Fri Sep 30 09:32:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:32:14.502733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694de8f1-0b1a-4f79-a6ce-8634c2c38a51 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:14.504253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-694de8f1-0b1a-4f79-a6ce-8634c2c38a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:14.555422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:14.610359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694de8f1-0b1a-4f79-a6ce-8634c2c38a51 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:14.684024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:14.684622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:14.684893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:14.685331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:14.688235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:14.784881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:14.786344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:14.789785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:14.804715 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 756/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:32:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:32:14.849363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e03a5-4a47-4f42-9e89-d95a074373a8 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:14.849679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e03a5-4a47-4f42-9e89-d95a074373a8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:14.849962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e03a5-4a47-4f42-9e89-d95a074373a8 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:14.851214 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 750/1506] 10.222.0.15 () {52 vars in 830 bytes} [Fri Sep 30 09:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:32:14.854759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:14.855960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:14.953290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2c334-c148-4108-98a2-dd31701867e0 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:15.046432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:15.047312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:15.047549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:15.047789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] PATH_INFO: `/v3/users/c06518a57bd546bf8cd1018d56604900` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:15.060441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06518a57bd546bf8cd1018d56604900)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:15.061477 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:32:15.061477 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Sep 30 09:32:15.061477 dv-cinder-master-3-smb devstack@keystone.service[81371]: Sep 30 09:32:15.062775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:15.072422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:15.132386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06518a57bd546bf8cd1018d56604900'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:15.146627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06518a57bd546bf8cd1018d56604900'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:15.147067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06518a57bd546bf8cd1018d56604900'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:15.147840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06518a57bd546bf8cd1018d56604900'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:15.158185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06518a57bd546bf8cd1018d56604900'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:15.187128 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 757/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:32:14 2022] DELETE /identity/v3/users/c06518a57bd546bf8cd1018d56604900 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:32:15.191624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbbcff-4786-4776-9ce1-e8425261101d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:15.192113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbbcff-4786-4776-9ce1-e8425261101d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:15.192211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbbcff-4786-4776-9ce1-e8425261101d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:15.248500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-05fbbcff-4786-4776-9ce1-e8425261101d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:32:15.249612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-05fbbcff-4786-4776-9ce1-e8425261101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:15.325242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:15.338275 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 751/1508] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:32:15.348645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:15.349904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:15.449615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560da299-81c1-4619-9366-b1775b31e500 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:15.503870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:15.524372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba3c54-f6fd-43eb-98ed-4c6d4da367b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:15.524986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba3c54-f6fd-43eb-98ed-4c6d4da367b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:15.525289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba3c54-f6fd-43eb-98ed-4c6d4da367b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:15.525570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba3c54-f6fd-43eb-98ed-4c6d4da367b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:15.527199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-75ba3c54-f6fd-43eb-98ed-4c6d4da367b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:15.634659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba3c54-f6fd-43eb-98ed-4c6d4da367b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:15.636141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba3c54-f6fd-43eb-98ed-4c6d4da367b4 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:15.654208 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 758/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:32:15.985906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbbcff-4786-4776-9ce1-e8425261101d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:32:15.987170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-05fbbcff-4786-4776-9ce1-e8425261101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:32:15.999826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbbcff-4786-4776-9ce1-e8425261101d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:32:16.098557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:32:16.099110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:32:16.099390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:32:16.099667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] PATH_INFO: `/v3/projects/bebe649190e34449981a61e5836e91b0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:32:16.110090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bebe649190e34449981a61e5836e91b0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:32:16.111112 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:32:16.111112 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Sep 30 09:32:16.111112 dv-cinder-master-3-smb devstack@keystone.service[81372]: Sep 30 09:32:16.112357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:32:16.170451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bebe649190e34449981a61e5836e91b0', 'request_id': 'req-0bca1f8c-21c0-48da-9854-f644c119c2d3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:16.180639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bebe649190e34449981a61e5836e91b0', 'request_id': 'req-0bca1f8c-21c0-48da-9854-f644c119c2d3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:16.196001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bebe649190e34449981a61e5836e91b0', 'request_id': 'req-0bca1f8c-21c0-48da-9854-f644c119c2d3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:16.217904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] The token cache is being invalidate because project bebe649190e34449981a61e5836e91b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:32:16.218193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:16.218624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0bca1f8c-21c0-48da-9854-f644c119c2d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:32:16.220111 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 752/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:32:15 2022] DELETE /identity/v3/projects/bebe649190e34449981a61e5836e91b0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:32:25.651884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=62 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:32:26.215996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:33:53.436100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f843b-4c85-4981-a8cf-b659449a1c33 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:33:53.436680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f843b-4c85-4981-a8cf-b659449a1c33 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:33:53.436680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f843b-4c85-4981-a8cf-b659449a1c33 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:33:53.438054 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:33:53.443132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6020-3861-4594-adfa-0d95bc7a6b17 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:33:53.443568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6020-3861-4594-adfa-0d95bc7a6b17 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:33:53.443674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd6020-3861-4594-adfa-0d95bc7a6b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:33:53.523858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e6dd6020-3861-4594-adfa-0d95bc7a6b17 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:33:53.525097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e6dd6020-3861-4594-adfa-0d95bc7a6b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:33:53.630474 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:33:53.787712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f843b-4c85-4981-a8cf-b659449a1c33 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:33:53.789828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-818f843b-4c85-4981-a8cf-b659449a1c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:33:53.906922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f843b-4c85-4981-a8cf-b659449a1c33 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:33:53.912339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83ac73-addc-4be8-8d76-657118c36dbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:33:53.912937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83ac73-addc-4be8-8d76-657118c36dbe service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:33:53.913219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83ac73-addc-4be8-8d76-657118c36dbe service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:33:53.913492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83ac73-addc-4be8-8d76-657118c36dbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:33:53.922238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83ac73-addc-4be8-8d76-657118c36dbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:33:53.923831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83ac73-addc-4be8-8d76-657118c36dbe service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:33:53.933781 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 760/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:33:55.006654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add8aba-b905-4781-8691-ca4f9d1afa49 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:33:55.007193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add8aba-b905-4781-8691-ca4f9d1afa49 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:33:55.007193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add8aba-b905-4781-8691-ca4f9d1afa49 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:33:55.008486 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:33:55.015271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3cfba-7a96-49a6-9af7-63bbd70ce9ea None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:33:55.015739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3cfba-7a96-49a6-9af7-63bbd70ce9ea None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:33:55.015839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3cfba-7a96-49a6-9af7-63bbd70ce9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:33:55.035391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-27b3cfba-7a96-49a6-9af7-63bbd70ce9ea None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:33:55.036605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-27b3cfba-7a96-49a6-9af7-63bbd70ce9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:33:55.045124 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:33:55.198901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add8aba-b905-4781-8691-ca4f9d1afa49 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:33:55.212787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add8aba-b905-4781-8691-ca4f9d1afa49 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:33:55.218225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540ee3a-7d94-43bf-9c9a-753e129a0a4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:33:55.218812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540ee3a-7d94-43bf-9c9a-753e129a0a4a service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:33:55.219088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540ee3a-7d94-43bf-9c9a-753e129a0a4a service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:33:55.219356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540ee3a-7d94-43bf-9c9a-753e129a0a4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:33:55.227120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540ee3a-7d94-43bf-9c9a-753e129a0a4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:33:55.228683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540ee3a-7d94-43bf-9c9a-753e129a0a4a service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:33:55.238421 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 755/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:33:55.337218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd151-cbee-41d9-846a-f8591020484e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:33:55.337504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd151-cbee-41d9-846a-f8591020484e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:33:55.337799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd151-cbee-41d9-846a-f8591020484e None None] PATH_INFO: `/v3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:33:55.339715 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 762/1517] 10.222.0.40 () {54 vars in 925 bytes} [Fri Sep 30 09:33:55 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:33:55.346576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439710b-a6a1-4dc5-a04d-35701aa97262 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:33:55.346855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439710b-a6a1-4dc5-a04d-35701aa97262 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:33:55.347244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439710b-a6a1-4dc5-a04d-35701aa97262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:33:55.411493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3439710b-a6a1-4dc5-a04d-35701aa97262 None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:33:55.412716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3439710b-a6a1-4dc5-a04d-35701aa97262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:33:55.429387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:33:55.485424 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 756/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Sep 30 09:33:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:34:04.460539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4160198-4598-4cba-a4eb-bddb1a36730c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:04.461015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4160198-4598-4cba-a4eb-bddb1a36730c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:04.461112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4160198-4598-4cba-a4eb-bddb1a36730c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:04.462449 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 763/1519] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 30 09:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:34:04.468021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfbe51-90fb-412a-821b-621a9cc30542 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:04.468426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfbe51-90fb-412a-821b-621a9cc30542 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:04.468582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfbe51-90fb-412a-821b-621a9cc30542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:04.488515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-35cfbe51-90fb-412a-821b-621a9cc30542 None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:34:04.489917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-35cfbe51-90fb-412a-821b-621a9cc30542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:04.499558 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 757/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 30 09:34:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:34:04.507038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4160198-4598-4cba-a4eb-bddb1a36730c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:34:04.520710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4160198-4598-4cba-a4eb-bddb1a36730c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:34:04.527044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18b149-2ff1-43ec-9616-c446b926ab9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:34:04.527651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18b149-2ff1-43ec-9616-c446b926ab9c service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:04.527935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18b149-2ff1-43ec-9616-c446b926ab9c service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:04.528207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18b149-2ff1-43ec-9616-c446b926ab9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:04.536364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18b149-2ff1-43ec-9616-c446b926ab9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:34:04.537998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18b149-2ff1-43ec-9616-c446b926ab9c service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:34:04.548608 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 764/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:34:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:34:06.729689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857679-7dfe-4dca-befe-de854495d640 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:06.730146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857679-7dfe-4dca-befe-de854495d640 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:06.730244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857679-7dfe-4dca-befe-de854495d640 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:06.731568 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:34:06.737553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8d81a-29f5-44e1-8fc2-2264d6538c41 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:06.738027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8d81a-29f5-44e1-8fc2-2264d6538c41 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:06.738272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8d81a-29f5-44e1-8fc2-2264d6538c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:06.757812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b3f8d81a-29f5-44e1-8fc2-2264d6538c41 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:34:06.759008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b3f8d81a-29f5-44e1-8fc2-2264d6538c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:06.767337 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:34:06.921210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857679-7dfe-4dca-befe-de854495d640 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:34:06.937734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857679-7dfe-4dca-befe-de854495d640 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:34:06.943123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d6bac-0e34-422c-9faf-31e1b51deba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:34:06.943720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d6bac-0e34-422c-9faf-31e1b51deba6 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:06.944000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d6bac-0e34-422c-9faf-31e1b51deba6 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:06.944273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d6bac-0e34-422c-9faf-31e1b51deba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:06.953092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d6bac-0e34-422c-9faf-31e1b51deba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:34:06.954721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d6bac-0e34-422c-9faf-31e1b51deba6 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:34:06.964052 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 759/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:34:08.058945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595c436-d1be-4a3b-be73-1fb61eba3e56 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:08.059496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595c436-d1be-4a3b-be73-1fb61eba3e56 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:08.059589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595c436-d1be-4a3b-be73-1fb61eba3e56 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:08.060894 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 30 09:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:34:08.066947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbbf9c-e5cf-4b63-986e-f5615b2f1f23 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:08.067411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbbf9c-e5cf-4b63-986e-f5615b2f1f23 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:08.067544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbbf9c-e5cf-4b63-986e-f5615b2f1f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:08.087176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3dcbbf9c-e5cf-4b63-986e-f5615b2f1f23 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:34:08.088966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3dcbbf9c-e5cf-4b63-986e-f5615b2f1f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:08.096740 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 30 09:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:34:08.253853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595c436-d1be-4a3b-be73-1fb61eba3e56 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:34:08.268173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595c436-d1be-4a3b-be73-1fb61eba3e56 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:34:08.273823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04188f-19ed-48c6-b53f-4a74c80123a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:34:08.274409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04188f-19ed-48c6-b53f-4a74c80123a8 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:08.274686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04188f-19ed-48c6-b53f-4a74c80123a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:08.274950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04188f-19ed-48c6-b53f-4a74c80123a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:08.283254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04188f-19ed-48c6-b53f-4a74c80123a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:34:08.284862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04188f-19ed-48c6-b53f-4a74c80123a8 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:34:08.294607 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 767/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:34:12.475983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10db9-b5b7-4ca2-8304-263928a936ff None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:12.476576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10db9-b5b7-4ca2-8304-263928a936ff None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:12.476576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db10db9-b5b7-4ca2-8304-263928a936ff None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:12.477896 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:34:12.483150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbf99-fbb0-46a9-8000-c84d65c0e019 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:12.483570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbf99-fbb0-46a9-8000-c84d65c0e019 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:12.483702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbf99-fbb0-46a9-8000-c84d65c0e019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:12.502993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b55cbf99-fbb0-46a9-8000-c84d65c0e019 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:34:12.504207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b55cbf99-fbb0-46a9-8000-c84d65c0e019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:12.512637 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:34:12.801931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db10db9-b5b7-4ca2-8304-263928a936ff None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:34:12.803629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5db10db9-b5b7-4ca2-8304-263928a936ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:12.880788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db10db9-b5b7-4ca2-8304-263928a936ff None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:34:12.884350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5835b2-7cf6-4b6e-b362-cac45aaa4596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:34:12.885523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5835b2-7cf6-4b6e-b362-cac45aaa4596 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:12.885829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5835b2-7cf6-4b6e-b362-cac45aaa4596 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:12.886095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5835b2-7cf6-4b6e-b362-cac45aaa4596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:12.895227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5835b2-7cf6-4b6e-b362-cac45aaa4596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:34:12.896693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5835b2-7cf6-4b6e-b362-cac45aaa4596 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:34:12.906785 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 762/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:34:15.047251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecfbac-dbf2-4a4d-8b58-d765816fd589 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:15.047793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecfbac-dbf2-4a4d-8b58-d765816fd589 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:15.047793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecfbac-dbf2-4a4d-8b58-d765816fd589 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:15.049123 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:34:15.054267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038780c6-d266-45fb-b937-1bb32bc93c90 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:15.054696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038780c6-d266-45fb-b937-1bb32bc93c90 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:15.054810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038780c6-d266-45fb-b937-1bb32bc93c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:15.076399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-038780c6-d266-45fb-b937-1bb32bc93c90 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:34:15.077594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-038780c6-d266-45fb-b937-1bb32bc93c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:15.085340 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:34:15.369434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecfbac-dbf2-4a4d-8b58-d765816fd589 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:34:15.380284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecfbac-dbf2-4a4d-8b58-d765816fd589 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:34:15.383823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4cd5be-02fa-4645-9d8a-817ff169587c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:34:15.384442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cd5be-02fa-4645-9d8a-817ff169587c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:15.385220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cd5be-02fa-4645-9d8a-817ff169587c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:15.385552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cd5be-02fa-4645-9d8a-817ff169587c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:15.394643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4cd5be-02fa-4645-9d8a-817ff169587c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:34:15.396182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4cd5be-02fa-4645-9d8a-817ff169587c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:34:15.406698 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 770/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:34:19.053289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f70f3-037b-4311-9549-78e7482dde1f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:19.053865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f70f3-037b-4311-9549-78e7482dde1f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:19.053865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f70f3-037b-4311-9549-78e7482dde1f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:19.055116 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:34:19.061144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924ef73-df63-43dc-89e9-72c832a55a71 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:19.061901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924ef73-df63-43dc-89e9-72c832a55a71 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:19.061901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924ef73-df63-43dc-89e9-72c832a55a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:19.069771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:34:19.083311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7924ef73-df63-43dc-89e9-72c832a55a71 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:34:19.084529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7924ef73-df63-43dc-89e9-72c832a55a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:19.093171 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:34:19.401487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f70f3-037b-4311-9549-78e7482dde1f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:34:19.403308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d02f70f3-037b-4311-9549-78e7482dde1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:19.467841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:34:19.481040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f70f3-037b-4311-9549-78e7482dde1f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:34:19.484639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91671201-990b-45f1-a201-5ef229eeacd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:34:19.485245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91671201-990b-45f1-a201-5ef229eeacd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:19.485528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91671201-990b-45f1-a201-5ef229eeacd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:19.485830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91671201-990b-45f1-a201-5ef229eeacd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:19.495289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91671201-990b-45f1-a201-5ef229eeacd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:34:19.496714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91671201-990b-45f1-a201-5ef229eeacd2 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:34:19.506001 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 765/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:34:21.694731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938c8b-56cd-44af-bdd6-80c5b64bb8d2 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:21.695262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938c8b-56cd-44af-bdd6-80c5b64bb8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:21.695262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e938c8b-56cd-44af-bdd6-80c5b64bb8d2 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:21.696557 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 30 09:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:34:21.702447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436739c-dfa8-43e8-8f79-9b17cda2a511 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:21.702907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436739c-dfa8-43e8-8f79-9b17cda2a511 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:21.703244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436739c-dfa8-43e8-8f79-9b17cda2a511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:21.722986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4436739c-dfa8-43e8-8f79-9b17cda2a511 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:34:21.724177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4436739c-dfa8-43e8-8f79-9b17cda2a511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:21.733231 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 30 09:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:34:22.037908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e938c8b-56cd-44af-bdd6-80c5b64bb8d2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:34:22.039581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e938c8b-56cd-44af-bdd6-80c5b64bb8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:34:22.052349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e938c8b-56cd-44af-bdd6-80c5b64bb8d2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:34:22.055783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167b850-35c4-45a5-a8e1-36327245a442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:34:22.056428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167b850-35c4-45a5-a8e1-36327245a442 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:34:22.056880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167b850-35c4-45a5-a8e1-36327245a442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:34:22.057159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167b850-35c4-45a5-a8e1-36327245a442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:34:22.065931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167b850-35c4-45a5-a8e1-36327245a442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:34:22.067341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167b850-35c4-45a5-a8e1-36327245a442 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:34:22.077211 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 773/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:34:31.718268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=11 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:34:32.072448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:54.521056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e20b2-5183-4056-a137-da592b7730f6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.523660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e20b2-5183-4056-a137-da592b7730f6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.523660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e20b2-5183-4056-a137-da592b7730f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.550513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b42e20b2-5183-4056-a137-da592b7730f6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:54.552152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b42e20b2-5183-4056-a137-da592b7730f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:54.564470 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 767/1540] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:54.574503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bf425-748a-43d6-9354-a39186bee9cf None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.575096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bf425-748a-43d6-9354-a39186bee9cf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.575096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bf425-748a-43d6-9354-a39186bee9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.605154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-949bf425-748a-43d6-9354-a39186bee9cf None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:54.608340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-949bf425-748a-43d6-9354-a39186bee9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:54.616525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63d6b2-7a98-40a9-8f2c-24f0905210d2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.617251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63d6b2-7a98-40a9-8f2c-24f0905210d2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.617973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63d6b2-7a98-40a9-8f2c-24f0905210d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.620095 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 774/1541] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:54.629398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bf425-748a-43d6-9354-a39186bee9cf None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:54.639108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bf425-748a-43d6-9354-a39186bee9cf None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:54.645177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ca63d6b2-7a98-40a9-8f2c-24f0905210d2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:54.651316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ca63d6b2-7a98-40a9-8f2c-24f0905210d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:54.654317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:54.655297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.655578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.655848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.656514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:35:54.656818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:54.658066 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:35:54.658066 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Sep 30 09:35:54.658066 dv-cinder-master-3-smb devstack@keystone.service[81371]: Sep 30 09:35:54.658924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:54.662713 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 768/1542] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:35:54.671473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0003-25e4-4441-b721-c13e5d7e6461 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.672404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0003-25e4-4441-b721-c13e5d7e6461 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.672924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0003-25e4-4441-b721-c13e5d7e6461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.679428 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 775/1543] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:35:54.689352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:54.712661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547a92d-47f5-4753-920c-fd903c071712 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:54.720672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4e4b0003-25e4-4441-b721-c13e5d7e6461 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:54.722303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4e4b0003-25e4-4441-b721-c13e5d7e6461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:54.729079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b6bad-b3e9-47db-9ac0-3b47645eac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:54.729780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6bad-b3e9-47db-9ac0-3b47645eac4a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.730065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6bad-b3e9-47db-9ac0-3b47645eac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.730344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b6bad-b3e9-47db-9ac0-3b47645eac4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.730996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b6bad-b3e9-47db-9ac0-3b47645eac4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:54.732940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b6bad-b3e9-47db-9ac0-3b47645eac4a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:54.748372 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 769/1544] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:35:54.753125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0003-25e4-4441-b721-c13e5d7e6461 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:54.772166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0003-25e4-4441-b721-c13e5d7e6461 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:54.790658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:54.791353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.791740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.792024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.792792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:35:54.793094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:54.800229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:54.805768 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 776/1545] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:54 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:35:54.807388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda06e9-077d-4fc6-8968-eff36a9eaff4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.807533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda06e9-077d-4fc6-8968-eff36a9eaff4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.807815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda06e9-077d-4fc6-8968-eff36a9eaff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.836566 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 770/1546] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:35:54.837280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:54.846439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ecda06e9-077d-4fc6-8968-eff36a9eaff4 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:54.847150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ecda06e9-077d-4fc6-8968-eff36a9eaff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:54.851503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b8fc6-8d5f-4fed-b3ab-1e01d0469f49 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:54.978045 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 777/1547] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:54.979362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda06e9-077d-4fc6-8968-eff36a9eaff4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:54.979616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a2a3b-0b04-4a57-8710-91988a1f9a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:54.980249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2a3b-0b04-4a57-8710-91988a1f9a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:54.980535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2a3b-0b04-4a57-8710-91988a1f9a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:54.980813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a2a3b-0b04-4a57-8710-91988a1f9a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:54.981450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a2a3b-0b04-4a57-8710-91988a1f9a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:54.983424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a2a3b-0b04-4a57-8710-91988a1f9a5e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.000815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda06e9-077d-4fc6-8968-eff36a9eaff4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.015275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8d334-b229-4f5b-951d-08ef743e9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.016377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8d334-b229-4f5b-951d-08ef743e9383 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.016875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8d334-b229-4f5b-951d-08ef743e9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.017384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8d334-b229-4f5b-951d-08ef743e9383 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.019247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8d334-b229-4f5b-951d-08ef743e9383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.021962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8d334-b229-4f5b-951d-08ef743e9383 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.045658 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 771/1548] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:54 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.046270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bb37c-ec22-438d-8e41-b3d174a59be5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.046563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bb37c-ec22-438d-8e41-b3d174a59be5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.046837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bb37c-ec22-438d-8e41-b3d174a59be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.075850 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 778/1549] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:54 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.076085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f52bb37c-ec22-438d-8e41-b3d174a59be5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.077419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eacda-009a-4e21-893c-f5195ddc385e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.077645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f52bb37c-ec22-438d-8e41-b3d174a59be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.077778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eacda-009a-4e21-893c-f5195ddc385e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.078024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eacda-009a-4e21-893c-f5195ddc385e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.111116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-863eacda-009a-4e21-893c-f5195ddc385e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.112363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-863eacda-009a-4e21-893c-f5195ddc385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.156397 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 772/1550] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.157675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e8f4-2067-4e8a-a79e-cc3e71ccf478 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.157951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e8f4-2067-4e8a-a79e-cc3e71ccf478 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.158220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e8f4-2067-4e8a-a79e-cc3e71ccf478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.177835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1d87e8f4-2067-4e8a-a79e-cc3e71ccf478 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.179047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1d87e8f4-2067-4e8a-a79e-cc3e71ccf478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.185486 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 779/1551] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.186360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eacda-009a-4e21-893c-f5195ddc385e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.187144 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 773/1552] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.188392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584071-a110-41b0-b722-313f137580d3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.188674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584071-a110-41b0-b722-313f137580d3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.189179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584071-a110-41b0-b722-313f137580d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.199671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eacda-009a-4e21-893c-f5195ddc385e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.204403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3eb216-0519-4aab-874f-0ac90c7ab3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.205012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3eb216-0519-4aab-874f-0ac90c7ab3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.205294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3eb216-0519-4aab-874f-0ac90c7ab3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.205567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3eb216-0519-4aab-874f-0ac90c7ab3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.206248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3eb216-0519-4aab-874f-0ac90c7ab3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.207889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3eb216-0519-4aab-874f-0ac90c7ab3a0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.208330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f8584071-a110-41b0-b722-313f137580d3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.209514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f8584071-a110-41b0-b722-313f137580d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.218085 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 774/1553] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.221943 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 780/1554] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:55.222090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584071-a110-41b0-b722-313f137580d3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.222193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bac41-b810-4c09-8620-2b121259722d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.222193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bac41-b810-4c09-8620-2b121259722d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.222193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bac41-b810-4c09-8620-2b121259722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.232661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584071-a110-41b0-b722-313f137580d3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.237321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef264220-ab87-4238-9e35-fa734f5a16c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.237959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef264220-ab87-4238-9e35-fa734f5a16c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.238243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef264220-ab87-4238-9e35-fa734f5a16c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.238513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef264220-ab87-4238-9e35-fa734f5a16c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.239138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef264220-ab87-4238-9e35-fa734f5a16c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.240624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-616bac41-b810-4c09-8620-2b121259722d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.241061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef264220-ab87-4238-9e35-fa734f5a16c8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.241973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-616bac41-b810-4c09-8620-2b121259722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.249988 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 781/1555] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.253255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906515-ab8c-44f3-b1ca-1a0bdaefb399 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.253818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906515-ab8c-44f3-b1ca-1a0bdaefb399 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.254273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906515-ab8c-44f3-b1ca-1a0bdaefb399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.267887 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 775/1556] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/users => generated 291 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.267887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44ac05-08df-4c70-95ed-4a71a15f0c51 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.267887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44ac05-08df-4c70-95ed-4a71a15f0c51 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.267887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44ac05-08df-4c70-95ed-4a71a15f0c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.275783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-69906515-ab8c-44f3-b1ca-1a0bdaefb399 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.276974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-69906515-ab8c-44f3-b1ca-1a0bdaefb399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.284898 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 782/1557] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.285706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69906515-ab8c-44f3-b1ca-1a0bdaefb399 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.288367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6f44ac05-08df-4c70-95ed-4a71a15f0c51 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.290261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6f44ac05-08df-4c70-95ed-4a71a15f0c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.298184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:55.299921 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 776/1558] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.301195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cca93-d7b6-4a30-a595-21a999bb4b25 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.301477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cca93-d7b6-4a30-a595-21a999bb4b25 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.301776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cca93-d7b6-4a30-a595-21a999bb4b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.302900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69906515-ab8c-44f3-b1ca-1a0bdaefb399 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.309360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.310083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.310364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.310631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.311277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:35:55.311574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.313282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.322290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-610cca93-d7b6-4a30-a595-21a999bb4b25 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.323488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-610cca93-d7b6-4a30-a595-21a999bb4b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.325370 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 783/1559] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:35:55.326206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.342331 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 777/1560] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.342331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145876-3571-4b15-9cbb-7ee9cf69b0c8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.342331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145876-3571-4b15-9cbb-7ee9cf69b0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.342331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145876-3571-4b15-9cbb-7ee9cf69b0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.342727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c1469-39f2-4950-a6af-73cc8465ca93 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.346207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8babc3c-5710-4411-ab61-0d9dc1a1fbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.346791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8babc3c-5710-4411-ab61-0d9dc1a1fbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.347070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8babc3c-5710-4411-ab61-0d9dc1a1fbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.347336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8babc3c-5710-4411-ab61-0d9dc1a1fbd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.347968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8babc3c-5710-4411-ab61-0d9dc1a1fbd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.349568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8babc3c-5710-4411-ab61-0d9dc1a1fbd7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.361063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ed145876-3571-4b15-9cbb-7ee9cf69b0c8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.361277 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 784/1561] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:55.362337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ed145876-3571-4b15-9cbb-7ee9cf69b0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.362517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa78617-aa94-480a-bb7b-ba2404f42a15 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.362789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa78617-aa94-480a-bb7b-ba2404f42a15 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.363055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa78617-aa94-480a-bb7b-ba2404f42a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.373791 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 778/1562] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.374966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed145876-3571-4b15-9cbb-7ee9cf69b0c8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.385594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1aa78617-aa94-480a-bb7b-ba2404f42a15 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.386843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1aa78617-aa94-480a-bb7b-ba2404f42a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.387253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed145876-3571-4b15-9cbb-7ee9cf69b0c8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.395821 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 785/1563] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.397140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945204e-c480-4b49-a31a-d97dbda49045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.397395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cb1fe-fd1f-4bb8-8ae9-62b24eeff033 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.397807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945204e-c480-4b49-a31a-d97dbda49045 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.397977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cb1fe-fd1f-4bb8-8ae9-62b24eeff033 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.398194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945204e-c480-4b49-a31a-d97dbda49045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.398345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cb1fe-fd1f-4bb8-8ae9-62b24eeff033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.398463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945204e-c480-4b49-a31a-d97dbda49045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.399031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945204e-c480-4b49-a31a-d97dbda49045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.400881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945204e-c480-4b49-a31a-d97dbda49045 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.408593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:55.415898 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 779/1564] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:55.415898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945204e-c480-4b49-a31a-d97dbda49045 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.418284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1f0cb1fe-fd1f-4bb8-8ae9-62b24eeff033 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.419475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1f0cb1fe-fd1f-4bb8-8ae9-62b24eeff033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.425734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945204e-c480-4b49-a31a-d97dbda49045 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.427559 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 786/1565] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.428278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cb1fe-fd1f-4bb8-8ae9-62b24eeff033 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.431775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.432396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.432679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.432952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.433597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:35:55.433940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.435544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.438180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cb1fe-fd1f-4bb8-8ae9-62b24eeff033 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.442623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47e801-f374-44c0-853b-a1a15e4c7804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.443204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47e801-f374-44c0-853b-a1a15e4c7804 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.443480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47e801-f374-44c0-853b-a1a15e4c7804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.443746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47e801-f374-44c0-853b-a1a15e4c7804 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.444363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47e801-f374-44c0-853b-a1a15e4c7804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.446899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47e801-f374-44c0-853b-a1a15e4c7804 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.448931 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 780/1566] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:35:55.449781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.462702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbb843-7e7b-4a9d-ac3f-27ceb58d74d9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.468475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.469081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.469363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.469650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.470315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:35:55.470622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.472331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.478483 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 787/1567] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.478483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47e801-f374-44c0-853b-a1a15e4c7804 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.495276 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 781/1568] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:35:55.495276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.498563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f85aa-b391-45f6-91a4-9a0ce30b26d2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.545859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47e801-f374-44c0-853b-a1a15e4c7804 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.550820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7eb4-28a6-4934-baf9-d1cbb1cf35e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.551689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7eb4-28a6-4934-baf9-d1cbb1cf35e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.552000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7eb4-28a6-4934-baf9-d1cbb1cf35e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.552394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb7eb4-28a6-4934-baf9-d1cbb1cf35e8 admin admin] PATH_INFO: `/v3/projects/1daa1564a5c643a5b62d96d0591f4e7f/users/6386fc0b0ece4ee1bedf1de609458166/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.553884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.555059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.555339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.555605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.556240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:35:55.556537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.558151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.568352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb7eb4-28a6-4934-baf9-d1cbb1cf35e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1daa1564a5c643a5b62d96d0591f4e7f, user_id=6386fc0b0ece4ee1bedf1de609458166, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.570746 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 782/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:35:55.571208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb7eb4-28a6-4934-baf9-d1cbb1cf35e8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.571703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.582750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4578-3a68-4380-b93f-783f25d12b32 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.584460 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 788/1570] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:55 2022] PUT /identity/v3/projects/1daa1564a5c643a5b62d96d0591f4e7f/users/6386fc0b0ece4ee1bedf1de609458166/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:55.585326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7eb4-28a6-4934-baf9-d1cbb1cf35e8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.640295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.640916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.641205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.641483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.642167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:35:55.642473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.644079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.654208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb7eb4-28a6-4934-baf9-d1cbb1cf35e8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.655772 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 783/1571] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:35:55.656847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.658931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cdc52-e7da-4048-aeff-b684bceea0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.659538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cdc52-e7da-4048-aeff-b684bceea0cc admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.659821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cdc52-e7da-4048-aeff-b684bceea0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.660274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cdc52-e7da-4048-aeff-b684bceea0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.660977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cdc52-e7da-4048-aeff-b684bceea0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.662616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cdc52-e7da-4048-aeff-b684bceea0cc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.668879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b21f62-f4d2-4b99-bb2a-5d32c38ad62b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.673523 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 789/1572] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:55.673752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.674304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.674593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.674868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.675055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cdc52-e7da-4048-aeff-b684bceea0cc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.675515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:35:55.675821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.677482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.685037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cdc52-e7da-4048-aeff-b684bceea0cc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.697355 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 784/1573] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:35:55.697539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d60a3-fa2f-4806-a1e8-e762f4b46e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.698079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.698201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d60a3-fa2f-4806-a1e8-e762f4b46e5f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.698201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d60a3-fa2f-4806-a1e8-e762f4b46e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.698201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d60a3-fa2f-4806-a1e8-e762f4b46e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.698201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d60a3-fa2f-4806-a1e8-e762f4b46e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.698201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d60a3-fa2f-4806-a1e8-e762f4b46e5f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.701526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2a1fb-6fe4-49d7-8bb0-7aa3bbba073d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.707000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b9399-e819-422c-9cef-2b984fc10994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.707597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9399-e819-422c-9cef-2b984fc10994 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.707877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9399-e819-422c-9cef-2b984fc10994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.708198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9399-e819-422c-9cef-2b984fc10994 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.708777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b9399-e819-422c-9cef-2b984fc10994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.712440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b9399-e819-422c-9cef-2b984fc10994 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.726676 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 790/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.726676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d60a3-fa2f-4806-a1e8-e762f4b46e5f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.729223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:55.736838 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 785/1575] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/users => generated 284 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.736838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b9399-e819-422c-9cef-2b984fc10994 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.747435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b9399-e819-422c-9cef-2b984fc10994 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.772503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:55.790745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d60a3-fa2f-4806-a1e8-e762f4b46e5f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.795450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e0af3-1219-4e6b-9a75-9c429dd45a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.796059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e0af3-1219-4e6b-9a75-9c429dd45a56 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.796341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e0af3-1219-4e6b-9a75-9c429dd45a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.796614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e0af3-1219-4e6b-9a75-9c429dd45a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.797248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e0af3-1219-4e6b-9a75-9c429dd45a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.799259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e0af3-1219-4e6b-9a75-9c429dd45a56 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.806615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.807230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.807505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.807773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.808853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.809915 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:35:55.809915 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Sep 30 09:35:55.809915 dv-cinder-master-3-smb devstack@keystone.service[81372]: Sep 30 09:35:55.811533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.826181 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 791/1576] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.827488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc151960-b9ba-4d79-83a7-45bcbca9d63d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.827781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc151960-b9ba-4d79-83a7-45bcbca9d63d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.828060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc151960-b9ba-4d79-83a7-45bcbca9d63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.844101 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 786/1577] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.845214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.857891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cc151960-b9ba-4d79-83a7-45bcbca9d63d None None] MFA Rules not processed for user `6386fc0b0ece4ee1bedf1de609458166`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:55.859116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cc151960-b9ba-4d79-83a7-45bcbca9d63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:55.909824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc350fd-6873-45e2-8e18-4519a7333ae3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.914887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946f86c-c825-457f-b68a-a660d9e6bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.915495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f86c-c825-457f-b68a-a660d9e6bd5f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.915776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f86c-c825-457f-b68a-a660d9e6bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.916068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f86c-c825-457f-b68a-a660d9e6bd5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.916823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946f86c-c825-457f-b68a-a660d9e6bd5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.918956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946f86c-c825-457f-b68a-a660d9e6bd5f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.922845 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 792/1578] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:55.923734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc151960-b9ba-4d79-83a7-45bcbca9d63d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.936254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc151960-b9ba-4d79-83a7-45bcbca9d63d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:55.941545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b235f0d-5a38-4911-882f-a2eb6d253811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:55.942392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235f0d-5a38-4911-882f-a2eb6d253811 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:55.942706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235f0d-5a38-4911-882f-a2eb6d253811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:55.943013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235f0d-5a38-4911-882f-a2eb6d253811 admin admin] PATH_INFO: `/v3/projects/a56dc2359b184f4c9ba8b34b38e8f293/users/48baba6e5a954fedbd024853e95eb38a/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:55.960131 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 787/1579] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:55.960131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946f86c-c825-457f-b68a-a660d9e6bd5f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:55.962696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b235f0d-5a38-4911-882f-a2eb6d253811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56dc2359b184f4c9ba8b34b38e8f293, user_id=48baba6e5a954fedbd024853e95eb38a, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:55.965784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b235f0d-5a38-4911-882f-a2eb6d253811 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:55.983258 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 793/1580] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:55 2022] PUT /identity/v3/projects/a56dc2359b184f4c9ba8b34b38e8f293/users/48baba6e5a954fedbd024853e95eb38a/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:55.984373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b235f0d-5a38-4911-882f-a2eb6d253811 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.033314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946f86c-c825-457f-b68a-a660d9e6bd5f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.035542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:56.047576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b235f0d-5a38-4911-882f-a2eb6d253811 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.052636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.053241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.053525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.053832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.054620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.055784 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:35:56.055784 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Sep 30 09:35:56.055784 dv-cinder-master-3-smb devstack@keystone.service[81371]: Sep 30 09:35:56.057010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.077994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:56.080998 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 794/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:56.082006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.092421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0cd04-9ab4-4827-a111-205011163217 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.096657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbf310-e562-4c2f-8c84-d764f196e416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.097004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80189d6-f679-453f-94db-6d7350afe648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.097279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbf310-e562-4c2f-8c84-d764f196e416 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.097560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbf310-e562-4c2f-8c84-d764f196e416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.097700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80189d6-f679-453f-94db-6d7350afe648 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.097887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbf310-e562-4c2f-8c84-d764f196e416 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.097987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80189d6-f679-453f-94db-6d7350afe648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.098180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80189d6-f679-453f-94db-6d7350afe648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.098499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbf310-e562-4c2f-8c84-d764f196e416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.098804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80189d6-f679-453f-94db-6d7350afe648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.100390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80189d6-f679-453f-94db-6d7350afe648 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.100498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbf310-e562-4c2f-8c84-d764f196e416 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.110936 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 795/1582] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.115118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80189d6-f679-453f-94db-6d7350afe648 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.125248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80189d6-f679-453f-94db-6d7350afe648 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.128217 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 788/1583] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:55 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:56.129202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbf310-e562-4c2f-8c84-d764f196e416 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.134038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dee030-777b-44c6-a3f8-45a3c459c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.134713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dee030-777b-44c6-a3f8-45a3c459c611 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.135063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dee030-777b-44c6-a3f8-45a3c459c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.135403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dee030-777b-44c6-a3f8-45a3c459c611 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.136101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dee030-777b-44c6-a3f8-45a3c459c611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.138155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dee030-777b-44c6-a3f8-45a3c459c611 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.164691 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 796/1584] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:56.164691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dee030-777b-44c6-a3f8-45a3c459c611 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.172735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dee030-777b-44c6-a3f8-45a3c459c611 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.200759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbf310-e562-4c2f-8c84-d764f196e416 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.205453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54901a-d89a-4221-a826-974c86c34b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.206096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54901a-d89a-4221-a826-974c86c34b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.206374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54901a-d89a-4221-a826-974c86c34b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.206642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54901a-d89a-4221-a826-974c86c34b6c admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.207269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54901a-d89a-4221-a826-974c86c34b6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.209180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54901a-d89a-4221-a826-974c86c34b6c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.235203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d8e10-a8f4-417e-ad92-d5ea32254ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.235868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d8e10-a8f4-417e-ad92-d5ea32254ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.236151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d8e10-a8f4-417e-ad92-d5ea32254ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.236430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d8e10-a8f4-417e-ad92-d5ea32254ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.236534 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 789/1585] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:56.237276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d8e10-a8f4-417e-ad92-d5ea32254ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.237711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f7fe-65e7-4d94-8b0f-8a6a3ebe4433 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.237997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f7fe-65e7-4d94-8b0f-8a6a3ebe4433 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.238272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f7fe-65e7-4d94-8b0f-8a6a3ebe4433 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.239599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d8e10-a8f4-417e-ad92-d5ea32254ae7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.239794 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 790/1586] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 30 09:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:35:56.240724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c1e70-52a7-4ac4-9e6f-bbde99214bf1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.241006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c1e70-52a7-4ac4-9e6f-bbde99214bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.241277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c1e70-52a7-4ac4-9e6f-bbde99214bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.267843 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 797/1587] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:56.267843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d8e10-a8f4-417e-ad92-d5ea32254ae7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.271955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0f0c1e70-52a7-4ac4-9e6f-bbde99214bf1 None None] MFA Rules not processed for user `48baba6e5a954fedbd024853e95eb38a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:56.273781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0f0c1e70-52a7-4ac4-9e6f-bbde99214bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:56.276770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d8e10-a8f4-417e-ad92-d5ea32254ae7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.281724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c92330-6f01-4d06-aa57-0e4e70c0809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.282324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92330-6f01-4d06-aa57-0e4e70c0809e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.282606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92330-6f01-4d06-aa57-0e4e70c0809e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.282883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92330-6f01-4d06-aa57-0e4e70c0809e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.283612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c92330-6f01-4d06-aa57-0e4e70c0809e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.285200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c92330-6f01-4d06-aa57-0e4e70c0809e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.295649 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 798/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.296455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c92330-6f01-4d06-aa57-0e4e70c0809e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.308073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c92330-6f01-4d06-aa57-0e4e70c0809e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.312590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80501977-8fbf-480a-92c9-75eae071cc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.313177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80501977-8fbf-480a-92c9-75eae071cc11 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.313848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80501977-8fbf-480a-92c9-75eae071cc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.313848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80501977-8fbf-480a-92c9-75eae071cc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.314366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80501977-8fbf-480a-92c9-75eae071cc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.316077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80501977-8fbf-480a-92c9-75eae071cc11 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.327201 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 799/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.327973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80501977-8fbf-480a-92c9-75eae071cc11 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.337925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80501977-8fbf-480a-92c9-75eae071cc11 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.342103 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 791/1590] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:56.342390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c048c-5d3d-422e-b120-193fa0e60950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.342959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c048c-5d3d-422e-b120-193fa0e60950 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.343136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c1e70-52a7-4ac4-9e6f-bbde99214bf1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.343244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c048c-5d3d-422e-b120-193fa0e60950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.343549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c048c-5d3d-422e-b120-193fa0e60950 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.344166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c048c-5d3d-422e-b120-193fa0e60950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.346117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c048c-5d3d-422e-b120-193fa0e60950 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.353537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c1e70-52a7-4ac4-9e6f-bbde99214bf1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.359760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93370732-38c0-4109-af20-b58f8a40f2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.360516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93370732-38c0-4109-af20-b58f8a40f2fd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.360960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93370732-38c0-4109-af20-b58f8a40f2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.361390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93370732-38c0-4109-af20-b58f8a40f2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.362100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93370732-38c0-4109-af20-b58f8a40f2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.364081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93370732-38c0-4109-af20-b58f8a40f2fd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.368948 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 800/1591] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:56.369864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c048c-5d3d-422e-b120-193fa0e60950 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.377229 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 792/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.378563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb910e-86ea-4ffc-8dd9-b3ba3c7bf1ee None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.378841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb910e-86ea-4ffc-8dd9-b3ba3c7bf1ee None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.379214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb910e-86ea-4ffc-8dd9-b3ba3c7bf1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.381493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c048c-5d3d-422e-b120-193fa0e60950 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.386388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21180d8a-fb4f-4269-96b5-158b4433d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.386992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21180d8a-fb4f-4269-96b5-158b4433d93f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.387256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21180d8a-fb4f-4269-96b5-158b4433d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.387524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21180d8a-fb4f-4269-96b5-158b4433d93f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.388193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21180d8a-fb4f-4269-96b5-158b4433d93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.389833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21180d8a-fb4f-4269-96b5-158b4433d93f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.400744 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 801/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.401388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21180d8a-fb4f-4269-96b5-158b4433d93f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.411603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21180d8a-fb4f-4269-96b5-158b4433d93f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.417982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d996f5-8c79-4c0d-9e48-7a1322915dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.418578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d996f5-8c79-4c0d-9e48-7a1322915dce admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.418863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d996f5-8c79-4c0d-9e48-7a1322915dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.419133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d996f5-8c79-4c0d-9e48-7a1322915dce admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.419771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d996f5-8c79-4c0d-9e48-7a1322915dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.422304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d996f5-8c79-4c0d-9e48-7a1322915dce admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.423692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-46bb910e-86ea-4ffc-8dd9-b3ba3c7bf1ee None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:56.424914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-46bb910e-86ea-4ffc-8dd9-b3ba3c7bf1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:56.434014 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 802/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.434897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d996f5-8c79-4c0d-9e48-7a1322915dce admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.444399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d996f5-8c79-4c0d-9e48-7a1322915dce admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.448896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:56.450119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcffcd-42cb-4324-a465-a592244adb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.450936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcffcd-42cb-4324-a465-a592244adb61 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.451620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcffcd-42cb-4324-a465-a592244adb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.451901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcffcd-42cb-4324-a465-a592244adb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.452601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcffcd-42cb-4324-a465-a592244adb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.454441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcffcd-42cb-4324-a465-a592244adb61 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.464350 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 803/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.468350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcffcd-42cb-4324-a465-a592244adb61 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.478330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcffcd-42cb-4324-a465-a592244adb61 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.482831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff004865-d213-4090-b0e1-4428e57f173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.483460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004865-d213-4090-b0e1-4428e57f173d admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.483748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004865-d213-4090-b0e1-4428e57f173d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.484021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff004865-d213-4090-b0e1-4428e57f173d admin admin] PATH_INFO: `/v3/projects/aea5ebf794614ae0aedabca7fc761b27/users/a04a0db0efeb4de887c99deaa7124e6b/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.498981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff004865-d213-4090-b0e1-4428e57f173d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea5ebf794614ae0aedabca7fc761b27, user_id=a04a0db0efeb4de887c99deaa7124e6b, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.499547 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 793/1596] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:56.500376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb910e-86ea-4ffc-8dd9-b3ba3c7bf1ee None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.501883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff004865-d213-4090-b0e1-4428e57f173d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.510993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb910e-86ea-4ffc-8dd9-b3ba3c7bf1ee None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.515103 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 804/1597] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:56 2022] PUT /identity/v3/projects/aea5ebf794614ae0aedabca7fc761b27/users/a04a0db0efeb4de887c99deaa7124e6b/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:56.526915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff004865-d213-4090-b0e1-4428e57f173d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.536931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:56.577015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b0f59-9be8-4b09-b3f3-32ee7f138279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.578145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b0f59-9be8-4b09-b3f3-32ee7f138279 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.578286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b0f59-9be8-4b09-b3f3-32ee7f138279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.578564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b0f59-9be8-4b09-b3f3-32ee7f138279 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.579534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b0f59-9be8-4b09-b3f3-32ee7f138279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.581601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b0f59-9be8-4b09-b3f3-32ee7f138279 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.598604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff004865-d213-4090-b0e1-4428e57f173d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.603876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45ff1f-e50f-4cda-8733-4a79721a7fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.604506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45ff1f-e50f-4cda-8733-4a79721a7fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.604859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45ff1f-e50f-4cda-8733-4a79721a7fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.605346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45ff1f-e50f-4cda-8733-4a79721a7fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.606112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45ff1f-e50f-4cda-8733-4a79721a7fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.607746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45ff1f-e50f-4cda-8733-4a79721a7fe0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.612949 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 794/1598] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:56.614323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b0f59-9be8-4b09-b3f3-32ee7f138279 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.620340 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 805/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.621179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45ff1f-e50f-4cda-8733-4a79721a7fe0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.631335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45ff1f-e50f-4cda-8733-4a79721a7fe0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.685564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b0f59-9be8-4b09-b3f3-32ee7f138279 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.690302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f9d50-3837-4a3d-af10-20160d66b4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.690910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f9d50-3837-4a3d-af10-20160d66b4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.691136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080c3fd-74ea-473e-8484-69efba6382c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.691279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f9d50-3837-4a3d-af10-20160d66b4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.691455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f9d50-3837-4a3d-af10-20160d66b4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.691763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080c3fd-74ea-473e-8484-69efba6382c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.692065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080c3fd-74ea-473e-8484-69efba6382c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.692175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f9d50-3837-4a3d-af10-20160d66b4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.692337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080c3fd-74ea-473e-8484-69efba6382c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.693016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080c3fd-74ea-473e-8484-69efba6382c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.693725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f9d50-3837-4a3d-af10-20160d66b4ad admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.694658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080c3fd-74ea-473e-8484-69efba6382c6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.703765 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 795/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.707355 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 806/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.707355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080c3fd-74ea-473e-8484-69efba6382c6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.707525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f9d50-3837-4a3d-af10-20160d66b4ad admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.717172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080c3fd-74ea-473e-8484-69efba6382c6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.717936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f9d50-3837-4a3d-af10-20160d66b4ad admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.721944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbedbf-8eb3-4f5a-865e-2c62ebc41d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.722580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbedbf-8eb3-4f5a-865e-2c62ebc41d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.722704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442d82-b3a6-4386-8fac-c24b7ff9d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.722856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbedbf-8eb3-4f5a-865e-2c62ebc41d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.723123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbedbf-8eb3-4f5a-865e-2c62ebc41d77 admin admin] PATH_INFO: `/v3/projects/732bb88e765b4fc3921cb3e37be0270a/users/c75799eec4744849a52d589e19424d6e/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.723293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442d82-b3a6-4386-8fac-c24b7ff9d789 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.723574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442d82-b3a6-4386-8fac-c24b7ff9d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.723846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442d82-b3a6-4386-8fac-c24b7ff9d789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.724482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442d82-b3a6-4386-8fac-c24b7ff9d789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.726699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442d82-b3a6-4386-8fac-c24b7ff9d789 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.737995 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 796/1602] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.738234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbedbf-8eb3-4f5a-865e-2c62ebc41d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=732bb88e765b4fc3921cb3e37be0270a, user_id=c75799eec4744849a52d589e19424d6e, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.738945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442d82-b3a6-4386-8fac-c24b7ff9d789 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.741049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbedbf-8eb3-4f5a-865e-2c62ebc41d77 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.756215 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 807/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:56 2022] PUT /identity/v3/projects/732bb88e765b4fc3921cb3e37be0270a/users/c75799eec4744849a52d589e19424d6e/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:56.757578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9fd8-c4e9-4235-b444-36047ee66590 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.758041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9fd8-c4e9-4235-b444-36047ee66590 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.758439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9fd8-c4e9-4235-b444-36047ee66590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.777320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:56.790258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-706d9fd8-c4e9-4235-b444-36047ee66590 None None] MFA Rules not processed for user `a04a0db0efeb4de887c99deaa7124e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:56.791674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-706d9fd8-c4e9-4235-b444-36047ee66590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:56.819476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442d82-b3a6-4386-8fac-c24b7ff9d789 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.858216 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 808/1604] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:56.859268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d9fd8-c4e9-4235-b444-36047ee66590 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.865473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:56.889887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57880a-b389-402f-9f55-6b5eef04939a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.890501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57880a-b389-402f-9f55-6b5eef04939a service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.890806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57880a-b389-402f-9f55-6b5eef04939a service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.891078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57880a-b389-402f-9f55-6b5eef04939a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.931275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d9fd8-c4e9-4235-b444-36047ee66590 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.935928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de9e3b-bfb4-41c0-a5aa-5b9e9933a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.936536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de9e3b-bfb4-41c0-a5aa-5b9e9933a052 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.936819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de9e3b-bfb4-41c0-a5aa-5b9e9933a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.937091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de9e3b-bfb4-41c0-a5aa-5b9e9933a052 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.937781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de9e3b-bfb4-41c0-a5aa-5b9e9933a052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.939716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de9e3b-bfb4-41c0-a5aa-5b9e9933a052 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.953072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57880a-b389-402f-9f55-6b5eef04939a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.954928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57880a-b389-402f-9f55-6b5eef04939a service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.965336 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 809/1605] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:56 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:56.965336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de9e3b-bfb4-41c0-a5aa-5b9e9933a052 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.965606 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 797/1606] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:56.965606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57880a-b389-402f-9f55-6b5eef04939a service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.975050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de9e3b-bfb4-41c0-a5aa-5b9e9933a052 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.976617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57880a-b389-402f-9f55-6b5eef04939a service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:56.980806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49479d-87c0-4b6f-adf3-03e39d0044d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.981375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:56.981740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49479d-87c0-4b6f-adf3-03e39d0044d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.982058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:56.982185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49479d-87c0-4b6f-adf3-03e39d0044d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.982332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:56.982525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49479d-87c0-4b6f-adf3-03e39d0044d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.982644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] PATH_INFO: `/v3/projects/383b9ec8f93443c58746f41a9b9142a9/users/9894741909ed48eabb9fe649951c6c25/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:56.983340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49479d-87c0-4b6f-adf3-03e39d0044d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:56.984971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49479d-87c0-4b6f-adf3-03e39d0044d1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:56.996610 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 810/1607] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:56.997439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49479d-87c0-4b6f-adf3-03e39d0044d1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:56.999070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383b9ec8f93443c58746f41a9b9142a9, user_id=9894741909ed48eabb9fe649951c6c25, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.000210 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:35:57.000210 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Sep 30 09:35:57.000210 dv-cinder-master-3-smb devstack@keystone.service[81372]: Sep 30 09:35:57.002289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.011000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49479d-87c0-4b6f-adf3-03e39d0044d1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.017760 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 798/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:56 2022] PUT /identity/v3/projects/383b9ec8f93443c58746f41a9b9142a9/users/9894741909ed48eabb9fe649951c6c25/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:57.022485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.068836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:57.085060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f996d-08cd-488b-a827-899c6efa7b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.085843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f996d-08cd-488b-a827-899c6efa7b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.086194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f996d-08cd-488b-a827-899c6efa7b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.086485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f996d-08cd-488b-a827-899c6efa7b4e admin admin] PATH_INFO: `/v3/projects/1a79b73c05184bb6b3586a71e6d2b31f/users/1907766086fd47ae824292f89a946342/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.093300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b25cc0-5e74-4d69-b63e-6ca1d1d24b16 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.098538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12fbbf-ef3f-4165-bfd6-feab645ab807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.099120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12fbbf-ef3f-4165-bfd6-feab645ab807 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.099412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12fbbf-ef3f-4165-bfd6-feab645ab807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.099715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12fbbf-ef3f-4165-bfd6-feab645ab807 admin admin] PATH_INFO: `/v3/projects/7258740d324b4be18675101826a78ca7/users/8efb5824b0c74435b41f3d7911c20874/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.102026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f996d-08cd-488b-a827-899c6efa7b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a79b73c05184bb6b3586a71e6d2b31f, user_id=1907766086fd47ae824292f89a946342, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.104698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f996d-08cd-488b-a827-899c6efa7b4e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.117394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12fbbf-ef3f-4165-bfd6-feab645ab807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7258740d324b4be18675101826a78ca7, user_id=8efb5824b0c74435b41f3d7911c20874, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.119046 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 811/1609] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:56 2022] PUT /identity/v3/projects/1a79b73c05184bb6b3586a71e6d2b31f/users/1907766086fd47ae824292f89a946342/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:57.120334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909a02-4bbc-4693-a5b0-929a9c5194e0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.120472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12fbbf-ef3f-4165-bfd6-feab645ab807 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.120658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909a02-4bbc-4693-a5b0-929a9c5194e0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.120901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909a02-4bbc-4693-a5b0-929a9c5194e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.137293 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 799/1610] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:57 2022] PUT /identity/v3/projects/7258740d324b4be18675101826a78ca7/users/8efb5824b0c74435b41f3d7911c20874/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:57.138379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12fbbf-ef3f-4165-bfd6-feab645ab807 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.144842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-98909a02-4bbc-4693-a5b0-929a9c5194e0 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:57.146086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-98909a02-4bbc-4693-a5b0-929a9c5194e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:57.202827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12fbbf-ef3f-4165-bfd6-feab645ab807 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.207566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89351b-81b1-4f38-8a4c-cbf0c4cf624e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.208273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89351b-81b1-4f38-8a4c-cbf0c4cf624e service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.208645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89351b-81b1-4f38-8a4c-cbf0c4cf624e service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.209013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89351b-81b1-4f38-8a4c-cbf0c4cf624e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.211712 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 812/1611] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:57.214529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98909a02-4bbc-4693-a5b0-929a9c5194e0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.228326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98909a02-4bbc-4693-a5b0-929a9c5194e0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.232986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa979442-63d7-4de2-b5dd-ab0c9d0943a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.233583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa979442-63d7-4de2-b5dd-ab0c9d0943a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.233904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa979442-63d7-4de2-b5dd-ab0c9d0943a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.234171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa979442-63d7-4de2-b5dd-ab0c9d0943a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.234807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa979442-63d7-4de2-b5dd-ab0c9d0943a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.236545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa979442-63d7-4de2-b5dd-ab0c9d0943a7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.237875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:57.249140 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 813/1612] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:57.250004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa979442-63d7-4de2-b5dd-ab0c9d0943a7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.260926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa979442-63d7-4de2-b5dd-ab0c9d0943a7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.265995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dae48-1673-4046-893c-7725d53bef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.266637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dae48-1673-4046-893c-7725d53bef4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.266919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dae48-1673-4046-893c-7725d53bef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.267195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dae48-1673-4046-893c-7725d53bef4f admin admin] PATH_INFO: `/v3/projects/277b0f36e38f42909ad307ca1d09b7da/users/e32778edb0e943f48e5f40453c97322b/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.272321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89351b-81b1-4f38-8a4c-cbf0c4cf624e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.274417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89351b-81b1-4f38-8a4c-cbf0c4cf624e service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.283565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dae48-1673-4046-893c-7725d53bef4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=277b0f36e38f42909ad307ca1d09b7da, user_id=e32778edb0e943f48e5f40453c97322b, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.284094 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 800/1613] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:57.286300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dae48-1673-4046-893c-7725d53bef4f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.286882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8b61-aa9e-4995-be66-0690095d236e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.287171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8b61-aa9e-4995-be66-0690095d236e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.287450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb8b61-aa9e-4995-be66-0690095d236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.303362 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 814/1614] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:57 2022] PUT /identity/v3/projects/277b0f36e38f42909ad307ca1d09b7da/users/e32778edb0e943f48e5f40453c97322b/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:57.304645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dede2-8554-4657-842d-f4cd5852b026 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.304927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dede2-8554-4657-842d-f4cd5852b026 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.305195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dede2-8554-4657-842d-f4cd5852b026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.322240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cebb8b61-aa9e-4995-be66-0690095d236e None None] MFA Rules not processed for user `9894741909ed48eabb9fe649951c6c25`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:57.323396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cebb8b61-aa9e-4995-be66-0690095d236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:57.334320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6b2dede2-8554-4657-842d-f4cd5852b026 None None] MFA Rules not processed for user `1907766086fd47ae824292f89a946342`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:57.336158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6b2dede2-8554-4657-842d-f4cd5852b026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:57.407809 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 801/1615] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:57.409597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73ad2-1c0f-4fb8-9915-92dd7d4ef090 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.409925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73ad2-1c0f-4fb8-9915-92dd7d4ef090 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.410202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d73ad2-1c0f-4fb8-9915-92dd7d4ef090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.416938 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 815/1616] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:57.418362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263cc3c-4365-477e-bad1-b8c4142f3e80 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.418667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263cc3c-4365-477e-bad1-b8c4142f3e80 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.418992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263cc3c-4365-477e-bad1-b8c4142f3e80 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.420249 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 816/1617] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 30 09:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:35:57.422431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263cc3c-4365-477e-bad1-b8c4142f3e80 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.433404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:57.448592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a9d73ad2-1c0f-4fb8-9915-92dd7d4ef090 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:57.450409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a9d73ad2-1c0f-4fb8-9915-92dd7d4ef090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:57.498111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263cc3c-4365-477e-bad1-b8c4142f3e80 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.501990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd8155-e6f5-49ad-beda-58c130663282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.502620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd8155-e6f5-49ad-beda-58c130663282 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.502983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd8155-e6f5-49ad-beda-58c130663282 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.503327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd8155-e6f5-49ad-beda-58c130663282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.519394 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 802/1618] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:57.520118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d73ad2-1c0f-4fb8-9915-92dd7d4ef090 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.530660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d73ad2-1c0f-4fb8-9915-92dd7d4ef090 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.535476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2635d-eff0-41b4-9cc5-635b03cf1052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.536175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2635d-eff0-41b4-9cc5-635b03cf1052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.536460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2635d-eff0-41b4-9cc5-635b03cf1052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.536738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2635d-eff0-41b4-9cc5-635b03cf1052 admin admin] PATH_INFO: `/v3/projects/a7018406563b46f48900b2fe70332e7c/users/8d8d82a3514f440e8d31c9cdc9877108/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.552415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2635d-eff0-41b4-9cc5-635b03cf1052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7018406563b46f48900b2fe70332e7c, user_id=8d8d82a3514f440e8d31c9cdc9877108, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.556816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2635d-eff0-41b4-9cc5-635b03cf1052 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.575274 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 803/1619] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:57 2022] PUT /identity/v3/projects/a7018406563b46f48900b2fe70332e7c/users/8d8d82a3514f440e8d31c9cdc9877108/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:57.576786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac2231-3513-437a-86f5-fc3ecfdabd6f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.577236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac2231-3513-437a-86f5-fc3ecfdabd6f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.578166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac2231-3513-437a-86f5-fc3ecfdabd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.586848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd8155-e6f5-49ad-beda-58c130663282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.588574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd8155-e6f5-49ad-beda-58c130663282 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.597501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-70ac2231-3513-437a-86f5-fc3ecfdabd6f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:57.598829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-70ac2231-3513-437a-86f5-fc3ecfdabd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:57.599475 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 817/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:57.600570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd8155-e6f5-49ad-beda-58c130663282 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.610893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd8155-e6f5-49ad-beda-58c130663282 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.623120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:57.681812 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 804/1621] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:57.684355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8857f-08d8-42da-8c7b-9efc99569a16 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.684645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8857f-08d8-42da-8c7b-9efc99569a16 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.685082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8857f-08d8-42da-8c7b-9efc99569a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.690630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675f6af-6d16-4d4e-8da9-c71adb8ba232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.691446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f6af-6d16-4d4e-8da9-c71adb8ba232 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.691832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f6af-6d16-4d4e-8da9-c71adb8ba232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.692226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675f6af-6d16-4d4e-8da9-c71adb8ba232 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.693296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675f6af-6d16-4d4e-8da9-c71adb8ba232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.698241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675f6af-6d16-4d4e-8da9-c71adb8ba232 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.708919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f5f8857f-08d8-42da-8c7b-9efc99569a16 None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:57.710269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f5f8857f-08d8-42da-8c7b-9efc99569a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:57.742964 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 818/1622] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:35:57 2022] POST /identity/v3/projects => generated 403 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:57.742964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477397f-fe9f-45fe-99c6-e8b94eef9470 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.742964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477397f-fe9f-45fe-99c6-e8b94eef9470 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.742964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477397f-fe9f-45fe-99c6-e8b94eef9470 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.744868 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 819/1623] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 30 09:35:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:35:57.751687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477397f-fe9f-45fe-99c6-e8b94eef9470 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.815434 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 805/1624] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:57.816560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8857f-08d8-42da-8c7b-9efc99569a16 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.827509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8857f-08d8-42da-8c7b-9efc99569a16 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.839029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:57.846650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477397f-fe9f-45fe-99c6-e8b94eef9470 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.850702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813caf1-a812-46a9-9c8f-e6408614e6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.852214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813caf1-a812-46a9-9c8f-e6408614e6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.852214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813caf1-a812-46a9-9c8f-e6408614e6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.852531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813caf1-a812-46a9-9c8f-e6408614e6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.928240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42ffe3-f369-44f4-a95b-ac42b765d696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.928862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42ffe3-f369-44f4-a95b-ac42b765d696 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.929138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42ffe3-f369-44f4-a95b-ac42b765d696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.930185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42ffe3-f369-44f4-a95b-ac42b765d696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.930865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42ffe3-f369-44f4-a95b-ac42b765d696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.932703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42ffe3-f369-44f4-a95b-ac42b765d696 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.944859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813caf1-a812-46a9-9c8f-e6408614e6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.947097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813caf1-a812-46a9-9c8f-e6408614e6c2 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:57.950080 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 806/1625] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:57.951414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42ffe3-f369-44f4-a95b-ac42b765d696 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.968533 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 820/1626] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:57.969441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813caf1-a812-46a9-9c8f-e6408614e6c2 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:57.974280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42ffe3-f369-44f4-a95b-ac42b765d696 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.980859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc19ca-5f8f-4a1f-ac35-e49553204298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.981162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813caf1-a812-46a9-9c8f-e6408614e6c2 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:57.982063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc19ca-5f8f-4a1f-ac35-e49553204298 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.982702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc19ca-5f8f-4a1f-ac35-e49553204298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.983210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc19ca-5f8f-4a1f-ac35-e49553204298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.987553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15abbeb-551b-4c85-b2ca-74bd609ded59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:57.988367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15abbeb-551b-4c85-b2ca-74bd609ded59 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:57.989502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15abbeb-551b-4c85-b2ca-74bd609ded59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:57.989992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15abbeb-551b-4c85-b2ca-74bd609ded59 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:57.991039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15abbeb-551b-4c85-b2ca-74bd609ded59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:57.995156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15abbeb-551b-4c85-b2ca-74bd609ded59 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:58.000688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc19ca-5f8f-4a1f-ac35-e49553204298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:58.010641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc19ca-5f8f-4a1f-ac35-e49553204298 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:58.024786 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 807/1627] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:58.026404 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 821/1628] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:35:57 2022] POST /identity/v3/users => generated 308 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:35:58.027968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15abbeb-551b-4c85-b2ca-74bd609ded59 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:58.029903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04500775-fcad-42da-b230-4317a604ce0f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:58.030371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04500775-fcad-42da-b230-4317a604ce0f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:58.030745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04500775-fcad-42da-b230-4317a604ce0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:58.065866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-04500775-fcad-42da-b230-4317a604ce0f None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:58.068002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-04500775-fcad-42da-b230-4317a604ce0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:58.082657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:58.126929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15abbeb-551b-4c85-b2ca-74bd609ded59 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:58.130878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2559a-6f62-4180-9738-7323e0827349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:58.131693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2559a-6f62-4180-9738-7323e0827349 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:58.132050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2559a-6f62-4180-9738-7323e0827349 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:58.132244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2559a-6f62-4180-9738-7323e0827349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:58.188004 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 808/1629] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 30 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:58.199802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04500775-fcad-42da-b230-4317a604ce0f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:58.212715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04500775-fcad-42da-b230-4317a604ce0f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:58.218963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3ef6d-0e2b-4f63-9652-1b5eb996ae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:58.219714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ef6d-0e2b-4f63-9652-1b5eb996ae1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:58.220061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ef6d-0e2b-4f63-9652-1b5eb996ae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:58.221524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ef6d-0e2b-4f63-9652-1b5eb996ae1b admin admin] PATH_INFO: `/v3/projects/a7018406563b46f48900b2fe70332e7c/users/8d8d82a3514f440e8d31c9cdc9877108/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:58.223056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3ef6d-0e2b-4f63-9652-1b5eb996ae1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7018406563b46f48900b2fe70332e7c, user_id=8d8d82a3514f440e8d31c9cdc9877108, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:58.227000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3ef6d-0e2b-4f63-9652-1b5eb996ae1b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:58.240566 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 809/1630] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:58 2022] PUT /identity/v3/projects/a7018406563b46f48900b2fe70332e7c/users/8d8d82a3514f440e8d31c9cdc9877108/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:58.241341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3ef6d-0e2b-4f63-9652-1b5eb996ae1b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:58.280694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2559a-6f62-4180-9738-7323e0827349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:58.285214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2559a-6f62-4180-9738-7323e0827349 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:58.313587 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 822/1631] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:58.326465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2559a-6f62-4180-9738-7323e0827349 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:58.352358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2559a-6f62-4180-9738-7323e0827349 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:58.415628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3ef6d-0e2b-4f63-9652-1b5eb996ae1b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:58.429815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b801bf9-b4b3-470c-b817-1b97de3e3587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:58.430679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801bf9-b4b3-470c-b817-1b97de3e3587 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:58.430964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801bf9-b4b3-470c-b817-1b97de3e3587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:58.431236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b801bf9-b4b3-470c-b817-1b97de3e3587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:58.431898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b801bf9-b4b3-470c-b817-1b97de3e3587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:58.435018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b801bf9-b4b3-470c-b817-1b97de3e3587 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:58.463352 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 810/1632] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:58.464689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51106-1953-4591-a09d-72bf4978a86b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:58.464978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51106-1953-4591-a09d-72bf4978a86b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:58.465252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb51106-1953-4591-a09d-72bf4978a86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:58.489546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43266b9c-cf2f-4bf5-89f0-60de73d59c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:58.492290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43266b9c-cf2f-4bf5-89f0-60de73d59c18 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:58.492728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43266b9c-cf2f-4bf5-89f0-60de73d59c18 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:58.493560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43266b9c-cf2f-4bf5-89f0-60de73d59c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:58.494529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:58.510258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ecb51106-1953-4591-a09d-72bf4978a86b None None] MFA Rules not processed for user `8d8d82a3514f440e8d31c9cdc9877108`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:58.512619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ecb51106-1953-4591-a09d-72bf4978a86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:58.654418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43266b9c-cf2f-4bf5-89f0-60de73d59c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:58.656049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43266b9c-cf2f-4bf5-89f0-60de73d59c18 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:58.683761 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 823/1633] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:58.697680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43266b9c-cf2f-4bf5-89f0-60de73d59c18 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:58.717806 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 811/1634] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:58.731754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51106-1953-4591-a09d-72bf4978a86b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:58.733872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43266b9c-cf2f-4bf5-89f0-60de73d59c18 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:58.751951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c421e-fe31-4386-84b8-31afa2c4da66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:58.754217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c421e-fe31-4386-84b8-31afa2c4da66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:58.755609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c421e-fe31-4386-84b8-31afa2c4da66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:58.759201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c421e-fe31-4386-84b8-31afa2c4da66 admin admin] PATH_INFO: `/v3/projects/14c923da33ce4a8ea9bd0bc6ed630f9d/users/84757d0dad9347cf9c199c9e1ea41d2a/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:58.764688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:58.808576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c421e-fe31-4386-84b8-31afa2c4da66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c923da33ce4a8ea9bd0bc6ed630f9d, user_id=84757d0dad9347cf9c199c9e1ea41d2a, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:58.821958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c421e-fe31-4386-84b8-31afa2c4da66 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:58.854205 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 824/1635] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:58 2022] PUT /identity/v3/projects/14c923da33ce4a8ea9bd0bc6ed630f9d/users/84757d0dad9347cf9c199c9e1ea41d2a/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:58.858350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c421e-fe31-4386-84b8-31afa2c4da66 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:58.864426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb51106-1953-4591-a09d-72bf4978a86b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:58.999608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832de20-8242-4c35-a12b-122cf7022449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:59.000445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832de20-8242-4c35-a12b-122cf7022449 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.000833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832de20-8242-4c35-a12b-122cf7022449 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.001207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832de20-8242-4c35-a12b-122cf7022449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.055179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c421e-fe31-4386-84b8-31afa2c4da66 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:59.066400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea1343-ee7a-4658-8838-a23191cf1c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:59.068474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea1343-ee7a-4658-8838-a23191cf1c68 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.070456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea1343-ee7a-4658-8838-a23191cf1c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.071065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea1343-ee7a-4658-8838-a23191cf1c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.071990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea1343-ee7a-4658-8838-a23191cf1c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:59.076585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea1343-ee7a-4658-8838-a23191cf1c68 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:59.103501 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 825/1636] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:35:59.105942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea1343-ee7a-4658-8838-a23191cf1c68 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:59.129326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea1343-ee7a-4658-8838-a23191cf1c68 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:59.141543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c5708-c6b8-4a97-8d0f-b97f9c51ed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:59.143384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5708-c6b8-4a97-8d0f-b97f9c51ed86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.143384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5708-c6b8-4a97-8d0f-b97f9c51ed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.145030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5708-c6b8-4a97-8d0f-b97f9c51ed86 admin admin] PATH_INFO: `/v3/projects/14c923da33ce4a8ea9bd0bc6ed630f9d/users/84757d0dad9347cf9c199c9e1ea41d2a/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.147693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832de20-8242-4c35-a12b-122cf7022449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:59.150802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c5708-c6b8-4a97-8d0f-b97f9c51ed86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c923da33ce4a8ea9bd0bc6ed630f9d, user_id=84757d0dad9347cf9c199c9e1ea41d2a, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:59.151702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832de20-8242-4c35-a12b-122cf7022449 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:59.159907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c5708-c6b8-4a97-8d0f-b97f9c51ed86 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:59.180714 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 812/1637] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:59.204078 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 826/1638] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:35:59 2022] PUT /identity/v3/projects/14c923da33ce4a8ea9bd0bc6ed630f9d/users/84757d0dad9347cf9c199c9e1ea41d2a/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:35:59.214769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600f27a-778c-4462-a710-8f8f4d0bb838 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.215207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600f27a-778c-4462-a710-8f8f4d0bb838 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.215764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600f27a-778c-4462-a710-8f8f4d0bb838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.220281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a57902-c44a-4fa5-a258-37e351d98004 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.220703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a57902-c44a-4fa5-a258-37e351d98004 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.221581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a57902-c44a-4fa5-a258-37e351d98004 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.224039 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 827/1639] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 30 09:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:35:59.230155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:59.266618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff023-a0cd-4faa-9bee-3f8d58dae27c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.267141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff023-a0cd-4faa-9bee-3f8d58dae27c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.267502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ff023-a0cd-4faa-9bee-3f8d58dae27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.309216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-629ff023-a0cd-4faa-9bee-3f8d58dae27c None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:59.315268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-629ff023-a0cd-4faa-9bee-3f8d58dae27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:59.322302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c600f27a-778c-4462-a710-8f8f4d0bb838 None None] MFA Rules not processed for user `84757d0dad9347cf9c199c9e1ea41d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:35:59.323954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c600f27a-778c-4462-a710-8f8f4d0bb838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:35:59.559113 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 813/1640] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:35:59 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:59.559987 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 828/1641] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:35:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:35:59.567664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600f27a-778c-4462-a710-8f8f4d0bb838 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:59.625003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ff023-a0cd-4faa-9bee-3f8d58dae27c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:59.653177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600f27a-778c-4462-a710-8f8f4d0bb838 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:59.665394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff2a28-734c-4b94-9232-a129c302b549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:59.665984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff2a28-734c-4b94-9232-a129c302b549 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.666877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff2a28-734c-4b94-9232-a129c302b549 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.666877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff2a28-734c-4b94-9232-a129c302b549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.753283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ff023-a0cd-4faa-9bee-3f8d58dae27c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:59.758280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db9606-e673-4306-86e2-aa3eea39059e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:59.765385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db9606-e673-4306-86e2-aa3eea39059e service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.765385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db9606-e673-4306-86e2-aa3eea39059e service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.765385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db9606-e673-4306-86e2-aa3eea39059e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.783061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db9606-e673-4306-86e2-aa3eea39059e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:59.784915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db9606-e673-4306-86e2-aa3eea39059e service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:59.790101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff2a28-734c-4b94-9232-a129c302b549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:59.793533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff2a28-734c-4b94-9232-a129c302b549 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:59.799399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:35:59.814874 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 829/1642] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:59 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:59.821369 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 814/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:35:59.890376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db9606-e673-4306-86e2-aa3eea39059e service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:35:59.900536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db9606-e673-4306-86e2-aa3eea39059e service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:35:59.905711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68688e5c-52bf-48d1-a6d1-7d2fd132e005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:35:59.906568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688e5c-52bf-48d1-a6d1-7d2fd132e005 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:35:59.907075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688e5c-52bf-48d1-a6d1-7d2fd132e005 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:35:59.907567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68688e5c-52bf-48d1-a6d1-7d2fd132e005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:35:59.926434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68688e5c-52bf-48d1-a6d1-7d2fd132e005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:35:59.928013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68688e5c-52bf-48d1-a6d1-7d2fd132e005 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:35:59.974234 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 830/1644] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:35:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:01.419161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff2a28-734c-4b94-9232-a129c302b549 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:01.523806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:01.572794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff2a28-734c-4b94-9232-a129c302b549 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:01.581965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:01.593690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:01.593690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:01.593690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] PATH_INFO: `/v3/users/9894741909ed48eabb9fe649951c6c25` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:01.593690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9894741909ed48eabb9fe649951c6c25)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:01.593690 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:36:01.593690 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Sep 30 09:36:01.593690 dv-cinder-master-3-smb devstack@keystone.service[81372]: Sep 30 09:36:01.595485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:01.773511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9894741909ed48eabb9fe649951c6c25'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:01.790978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9894741909ed48eabb9fe649951c6c25'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:01.792444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9894741909ed48eabb9fe649951c6c25'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:01.793388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9894741909ed48eabb9fe649951c6c25'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:01.812052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9894741909ed48eabb9fe649951c6c25'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:01.869618 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 815/1645] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:01 2022] DELETE /identity/v3/users/9894741909ed48eabb9fe649951c6c25 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:01.876182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cba44b-2834-4ccd-9c37-5ce80f14da60 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:01.877224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cba44b-2834-4ccd-9c37-5ce80f14da60 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:01.877657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cba44b-2834-4ccd-9c37-5ce80f14da60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:01.915866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-34cba44b-2834-4ccd-9c37-5ce80f14da60 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:01.921559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-34cba44b-2834-4ccd-9c37-5ce80f14da60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:02.052478 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 831/1646] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:02.068428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:02.070771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:02.256927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9fe42-64ae-4443-8058-41488efef4e7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:02.274810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d65617-7b1c-4f21-9346-4d1e747fccba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:02.275604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d65617-7b1c-4f21-9346-4d1e747fccba service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:02.278790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d65617-7b1c-4f21-9346-4d1e747fccba service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:02.279213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d65617-7b1c-4f21-9346-4d1e747fccba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:02.305332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d65617-7b1c-4f21-9346-4d1e747fccba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:02.306967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d65617-7b1c-4f21-9346-4d1e747fccba service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:02.332737 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 816/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:03.011346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cba44b-2834-4ccd-9c37-5ce80f14da60 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:03.013429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-34cba44b-2834-4ccd-9c37-5ce80f14da60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:03.030334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cba44b-2834-4ccd-9c37-5ce80f14da60 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:03.058792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:03.059729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:03.060228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:03.060705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] PATH_INFO: `/v3/users/6386fc0b0ece4ee1bedf1de609458166` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:03.066775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6386fc0b0ece4ee1bedf1de609458166)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:03.069392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:03.191888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6386fc0b0ece4ee1bedf1de609458166'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:03.211554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6386fc0b0ece4ee1bedf1de609458166'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:03.212415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6386fc0b0ece4ee1bedf1de609458166'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:03.214492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6386fc0b0ece4ee1bedf1de609458166'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:03.243279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6386fc0b0ece4ee1bedf1de609458166'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:03.274765 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 832/1648] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:03 2022] DELETE /identity/v3/users/6386fc0b0ece4ee1bedf1de609458166 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:03.279184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c1ad2-a203-438a-bd1d-fa52f7f23559 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:03.279648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c1ad2-a203-438a-bd1d-fa52f7f23559 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:03.279731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c1ad2-a203-438a-bd1d-fa52f7f23559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:03.347776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6a5c1ad2-a203-438a-bd1d-fa52f7f23559 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:03.348947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6a5c1ad2-a203-438a-bd1d-fa52f7f23559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:03.513034 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 817/1649] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:03.747903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:03.750116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:03.912097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9270cc-6a8a-4514-b71c-cd1147f7c6cd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:03.941058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef42c328-1a79-4eb3-a5ce-04546a04e077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:03.941897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42c328-1a79-4eb3-a5ce-04546a04e077 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:03.942251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42c328-1a79-4eb3-a5ce-04546a04e077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:03.954227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef42c328-1a79-4eb3-a5ce-04546a04e077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:03.980801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42c328-1a79-4eb3-a5ce-04546a04e077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:03.984491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef42c328-1a79-4eb3-a5ce-04546a04e077 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:04.003627 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 833/1650] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:05.257760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c1ad2-a203-438a-bd1d-fa52f7f23559 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:05.260022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6a5c1ad2-a203-438a-bd1d-fa52f7f23559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:05.272173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:05.293435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c1ad2-a203-438a-bd1d-fa52f7f23559 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:05.309012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:05.310901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:05.312356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:05.312744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] PATH_INFO: `/v3/projects/1daa1564a5c643a5b62d96d0591f4e7f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:05.316475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1daa1564a5c643a5b62d96d0591f4e7f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:05.322246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:05.490216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1daa1564a5c643a5b62d96d0591f4e7f', 'request_id': 'req-f3c4f813-8e7a-47e7-adb2-cc37687f2412'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:05.534179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1daa1564a5c643a5b62d96d0591f4e7f', 'request_id': 'req-f3c4f813-8e7a-47e7-adb2-cc37687f2412'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:05.570451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1daa1564a5c643a5b62d96d0591f4e7f', 'request_id': 'req-f3c4f813-8e7a-47e7-adb2-cc37687f2412'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:05.603620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] The token cache is being invalidate because project 1daa1564a5c643a5b62d96d0591f4e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:05.608734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:05.612897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:05.615299 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 818/1651] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:05 2022] DELETE /identity/v3/projects/1daa1564a5c643a5b62d96d0591f4e7f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:05.621118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef39ad-9612-43b6-87e7-c40f2f50f80b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:05.622221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef39ad-9612-43b6-87e7-c40f2f50f80b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:05.622651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef39ad-9612-43b6-87e7-c40f2f50f80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:05.634124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:05.659576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-75ef39ad-9612-43b6-87e7-c40f2f50f80b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:05.661892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-75ef39ad-9612-43b6-87e7-c40f2f50f80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:05.815167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:05.819035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:05.889715 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 834/1652] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:05.902355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9872f-d644-4066-9af7-573138211977 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:05.914280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9872f-d644-4066-9af7-573138211977 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:05.914859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9872f-d644-4066-9af7-573138211977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:05.982549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-45d9872f-d644-4066-9af7-573138211977 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:05.992492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-45d9872f-d644-4066-9af7-573138211977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:06.021072 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 835/1653] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:06.025236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9872f-d644-4066-9af7-573138211977 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:06.056015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9872f-d644-4066-9af7-573138211977 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:06.073278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4f813-8e7a-47e7-adb2-cc37687f2412 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:06.079020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:06.079802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.080300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.080688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.082202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:06.082645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:06.084669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:06.094149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:06.095921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.102445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.102445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] PATH_INFO: `/v3/projects/383b9ec8f93443c58746f41a9b9142a9` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.110677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] RBAC: Authorizing `identity:delete_project(project_id=383b9ec8f93443c58746f41a9b9142a9)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:06.114938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:06.129380 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 836/1654] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:06.134648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:06.164485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da77c77-a516-4149-a90f-133d7bd3c385 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:06.189382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:06.190208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.190735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.191428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.192403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:06.195934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:06.234481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383b9ec8f93443c58746f41a9b9142a9', 'request_id': 'req-338f8363-5bda-4b30-8dc7-e5d18f30373f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.249285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383b9ec8f93443c58746f41a9b9142a9', 'request_id': 'req-338f8363-5bda-4b30-8dc7-e5d18f30373f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.261368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383b9ec8f93443c58746f41a9b9142a9', 'request_id': 'req-338f8363-5bda-4b30-8dc7-e5d18f30373f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.310008 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 837/1655] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:06 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:06.312100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] The token cache is being invalidate because project 383b9ec8f93443c58746f41a9b9142a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:06.312590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.313593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:06.316368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:06.317221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.331424 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 819/1656] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:05 2022] DELETE /identity/v3/projects/383b9ec8f93443c58746f41a9b9142a9 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:06.335125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:06.350184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:06.409059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:06.567233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f8363-5bda-4b30-8dc7-e5d18f30373f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:06.573227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb57dd-1d37-421a-adfa-e65a88eafae7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:06.581427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:06.581810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b30253-db4a-4afd-beaa-d550944c2014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:06.583162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b30253-db4a-4afd-beaa-d550944c2014 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.583497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b30253-db4a-4afd-beaa-d550944c2014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.583840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b30253-db4a-4afd-beaa-d550944c2014 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.584701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b30253-db4a-4afd-beaa-d550944c2014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:06.586269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.586808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.587993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] PATH_INFO: `/v3/users/84757d0dad9347cf9c199c9e1ea41d2a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.588114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b30253-db4a-4afd-beaa-d550944c2014 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:06.591470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=84757d0dad9347cf9c199c9e1ea41d2a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:06.594277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:06.627746 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 838/1657] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:06 2022] POST /identity/v3/users => generated 283 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:06.631104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b30253-db4a-4afd-beaa-d550944c2014 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:06.648755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b30253-db4a-4afd-beaa-d550944c2014 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:06.656507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eb466-21c2-4d71-92b2-7f465c14432d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:06.657156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb466-21c2-4d71-92b2-7f465c14432d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.657839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb466-21c2-4d71-92b2-7f465c14432d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.658242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb466-21c2-4d71-92b2-7f465c14432d admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.659537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eb466-21c2-4d71-92b2-7f465c14432d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:06.662109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eb466-21c2-4d71-92b2-7f465c14432d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:06.686710 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 839/1658] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:06.689364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eb466-21c2-4d71-92b2-7f465c14432d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:06.707658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eb466-21c2-4d71-92b2-7f465c14432d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:06.717707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fcaa-4b08-400f-b873-be2b55fc17ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:06.718622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fcaa-4b08-400f-b873-be2b55fc17ec admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.719047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fcaa-4b08-400f-b873-be2b55fc17ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.719464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fcaa-4b08-400f-b873-be2b55fc17ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.720408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560fcaa-4b08-400f-b873-be2b55fc17ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:06.723569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560fcaa-4b08-400f-b873-be2b55fc17ec admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:06.730853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84757d0dad9347cf9c199c9e1ea41d2a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.752656 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 840/1659] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:06.754165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84757d0dad9347cf9c199c9e1ea41d2a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.754590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fcaa-4b08-400f-b873-be2b55fc17ec admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:06.755044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84757d0dad9347cf9c199c9e1ea41d2a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.755776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84757d0dad9347cf9c199c9e1ea41d2a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.768734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d3a60d1d-c16b-43cb-8255-79e061f3b8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84757d0dad9347cf9c199c9e1ea41d2a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:06.770333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fcaa-4b08-400f-b873-be2b55fc17ec admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:06.779198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:06.780111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.780700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.781135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] PATH_INFO: `/v3/projects/bb6c2450405e46c3b4b30cb58934e123/users/5b9c41ae78f941b7ad698693f29c4a3d/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.821074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6c2450405e46c3b4b30cb58934e123, user_id=5b9c41ae78f941b7ad698693f29c4a3d, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:06.821573 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 820/1660] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:06 2022] DELETE /identity/v3/users/84757d0dad9347cf9c199c9e1ea41d2a => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:06.827284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:06.883119 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 841/1661] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:06 2022] PUT /identity/v3/projects/bb6c2450405e46c3b4b30cb58934e123/users/5b9c41ae78f941b7ad698693f29c4a3d/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:06.895087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5cf2e-8a6a-4c5d-9de1-25b4b237af5f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:06.895373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5cf2e-8a6a-4c5d-9de1-25b4b237af5f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:06.895887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5cf2e-8a6a-4c5d-9de1-25b4b237af5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:06.957206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d5a5cf2e-8a6a-4c5d-9de1-25b4b237af5f None None] MFA Rules not processed for user `5b9c41ae78f941b7ad698693f29c4a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:06.959792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d5a5cf2e-8a6a-4c5d-9de1-25b4b237af5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:07.149995 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 821/1662] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:07.154629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:07.156488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:07.165216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:07.295110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832cad4-0f0a-47ad-b034-119a00d51d67 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:07.308107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43531b8-7010-4c00-a601-e5ccea920d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:07.311765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43531b8-7010-4c00-a601-e5ccea920d1b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:07.311765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43531b8-7010-4c00-a601-e5ccea920d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:07.312048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43531b8-7010-4c00-a601-e5ccea920d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:07.313778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43531b8-7010-4c00-a601-e5ccea920d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:07.318699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43531b8-7010-4c00-a601-e5ccea920d1b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:07.384869 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 842/1663] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:07 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:07.388722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5cf2e-8a6a-4c5d-9de1-25b4b237af5f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:07.426378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5cf2e-8a6a-4c5d-9de1-25b4b237af5f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:07.511530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:07.631739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae3102-7f5b-4c81-a3c9-83037c32a1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:07.633288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae3102-7f5b-4c81-a3c9-83037c32a1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:07.633618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae3102-7f5b-4c81-a3c9-83037c32a1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:07.633925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae3102-7f5b-4c81-a3c9-83037c32a1f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:07.634922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae3102-7f5b-4c81-a3c9-83037c32a1f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:07.636657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae3102-7f5b-4c81-a3c9-83037c32a1f1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:07.706906 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 822/1664] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:07 2022] POST /identity/v3/users => generated 289 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:07.708941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43531b8-7010-4c00-a601-e5ccea920d1b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:07.752606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43531b8-7010-4c00-a601-e5ccea920d1b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:07.779371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:07.780116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:07.780507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:07.780887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:07.781670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:07.783403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:07.830354 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 843/1665] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:07.834513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae3102-7f5b-4c81-a3c9-83037c32a1f1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:07.852960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae3102-7f5b-4c81-a3c9-83037c32a1f1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:07.867792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54308-cd79-445c-a6f6-6c88dca73fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:07.868549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54308-cd79-445c-a6f6-6c88dca73fff admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:07.868753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54308-cd79-445c-a6f6-6c88dca73fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:07.868999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54308-cd79-445c-a6f6-6c88dca73fff admin admin] PATH_INFO: `/v3/projects/8b05fd3ff10c4d66bdfb3446ab709604/users/0af61f60c0ea4725bcddbd311734946a/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:07.874974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:07.876747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:07.907203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12344f83-5be9-4b68-8272-31ed63502028 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:07.941744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef54308-cd79-445c-a6f6-6c88dca73fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b05fd3ff10c4d66bdfb3446ab709604, user_id=0af61f60c0ea4725bcddbd311734946a, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:07.948801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:07.951035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:07.951280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:07.963040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef54308-cd79-445c-a6f6-6c88dca73fff admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:07.965514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] PATH_INFO: `/v3/users/1907766086fd47ae824292f89a946342` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:07.969012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1907766086fd47ae824292f89a946342)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:07.978051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:08.012844 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 823/1666] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:07 2022] PUT /identity/v3/projects/8b05fd3ff10c4d66bdfb3446ab709604/users/0af61f60c0ea4725bcddbd311734946a/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:08.020317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54308-cd79-445c-a6f6-6c88dca73fff admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:08.093700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1907766086fd47ae824292f89a946342'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:08.103950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1907766086fd47ae824292f89a946342'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:08.104981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1907766086fd47ae824292f89a946342'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:08.106586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1907766086fd47ae824292f89a946342'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:08.140856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-852e6e6e-635f-4933-889e-b9addda45006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1907766086fd47ae824292f89a946342'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:08.229295 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 844/1667] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:07 2022] DELETE /identity/v3/users/1907766086fd47ae824292f89a946342 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:08.246295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c9bf-960d-4363-9c9b-e0baa15e0988 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:08.247726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54308-cd79-445c-a6f6-6c88dca73fff admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:08.248271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c9bf-960d-4363-9c9b-e0baa15e0988 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:08.250691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c9bf-960d-4363-9c9b-e0baa15e0988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:08.332823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3e53c9bf-960d-4363-9c9b-e0baa15e0988 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:08.336183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e53c9bf-960d-4363-9c9b-e0baa15e0988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:08.442222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:08.442864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:08.443297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:08.443713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:08.444563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:08.449288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:08.477705 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 824/1668] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:08.478905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:08.486338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:08.499232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:08.517262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06c022-d100-4e3a-a565-99a83d3387b3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:08.529657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:08.531758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:08.532268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:08.532760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] PATH_INFO: `/v3/projects/8b05fd3ff10c4d66bdfb3446ab709604/users/0af61f60c0ea4725bcddbd311734946a/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:08.538629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b05fd3ff10c4d66bdfb3446ab709604, user_id=0af61f60c0ea4725bcddbd311734946a, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:08.543239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:08.556379 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 845/1669] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:08.584824 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 825/1670] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:08 2022] PUT /identity/v3/projects/8b05fd3ff10c4d66bdfb3446ab709604/users/0af61f60c0ea4725bcddbd311734946a/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:08.587371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbce90-d925-49a2-b839-abf66233818f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:08.587744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbce90-d925-49a2-b839-abf66233818f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:08.587968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbce90-d925-49a2-b839-abf66233818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:08.667196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7cfbce90-d925-49a2-b839-abf66233818f None None] MFA Rules not processed for user `0af61f60c0ea4725bcddbd311734946a`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:08.668469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7cfbce90-d925-49a2-b839-abf66233818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:08.674470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:08.680090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:08.773470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:08.820461 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 846/1671] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:08.832882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbce90-d925-49a2-b839-abf66233818f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:08.834595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7cfbce90-d925-49a2-b839-abf66233818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:08.849022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8237d1-873c-41fa-a87a-82f70b6351c4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:08.857889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:08.858539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:08.859650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:08.859650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:08.969139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbce90-d925-49a2-b839-abf66233818f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:08.973247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:08.973976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:08.974256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:08.974530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:08.988805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:08.990237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:08.993086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:08.996021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:09.008209 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 847/1672] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:09.043898 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 826/1673] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:09.203921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:09.205528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:09.228683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5c94f-65cd-4e37-8352-1573d52edf50 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:09.239150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c060d-367f-445d-83d7-31e20f795332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:09.242914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c060d-367f-445d-83d7-31e20f795332 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:09.243212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c060d-367f-445d-83d7-31e20f795332 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:09.243489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c060d-367f-445d-83d7-31e20f795332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:09.249464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3f3c060d-367f-445d-83d7-31e20f795332 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:09.400177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c060d-367f-445d-83d7-31e20f795332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:09.406123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c060d-367f-445d-83d7-31e20f795332 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:09.432249 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 848/1674] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:09.604064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:09.606385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:09.658980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fac6fd-ed1e-4aab-a3d8-8b3b1c6e3d63 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:09.675328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:09.680220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:09.681022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:09.682342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] PATH_INFO: `/v3/projects/14c923da33ce4a8ea9bd0bc6ed630f9d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:09.684722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c923da33ce4a8ea9bd0bc6ed630f9d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:09.688603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:09.858244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c923da33ce4a8ea9bd0bc6ed630f9d', 'request_id': 'req-e604684d-ebaa-4c5b-a55c-f39ac43b1040'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:09.871369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c923da33ce4a8ea9bd0bc6ed630f9d', 'request_id': 'req-e604684d-ebaa-4c5b-a55c-f39ac43b1040'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:09.892423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c923da33ce4a8ea9bd0bc6ed630f9d', 'request_id': 'req-e604684d-ebaa-4c5b-a55c-f39ac43b1040'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:09.921335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] The token cache is being invalidate because project 14c923da33ce4a8ea9bd0bc6ed630f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:09.921733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:09.922512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e604684d-ebaa-4c5b-a55c-f39ac43b1040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:09.924422 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 827/1675] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:09 2022] DELETE /identity/v3/projects/14c923da33ce4a8ea9bd0bc6ed630f9d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:09.929919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156eba49-244e-4f27-8bf1-99439f951081 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:09.930535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156eba49-244e-4f27-8bf1-99439f951081 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:09.930719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156eba49-244e-4f27-8bf1-99439f951081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:09.951942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7fa94-b53c-48ca-9473-e5b550892ef2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:09.952536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7fa94-b53c-48ca-9473-e5b550892ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:09.955530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7fa94-b53c-48ca-9473-e5b550892ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:10.002534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-156eba49-244e-4f27-8bf1-99439f951081 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:10.004293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-156eba49-244e-4f27-8bf1-99439f951081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.039579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b1f7fa94-b53c-48ca-9473-e5b550892ef2 None None] MFA Rules not processed for user `e32778edb0e943f48e5f40453c97322b`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:10.044416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:10.044725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b1f7fa94-b53c-48ca-9473-e5b550892ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.155722 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 849/1676] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:10.156794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156eba49-244e-4f27-8bf1-99439f951081 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:10.157728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:10.159852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-156eba49-244e-4f27-8bf1-99439f951081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.184268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156eba49-244e-4f27-8bf1-99439f951081 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:10.205468 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 828/1677] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:10.205697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:10.205805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7fa94-b53c-48ca-9473-e5b550892ef2 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:10.205885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:10.205885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:10.205885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] PATH_INFO: `/v3/users/48baba6e5a954fedbd024853e95eb38a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:10.206111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b1f7fa94-b53c-48ca-9473-e5b550892ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.207239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] RBAC: Authorizing `identity:delete_user(user_id=48baba6e5a954fedbd024853e95eb38a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:10.209311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:10.228351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7fa94-b53c-48ca-9473-e5b550892ef2 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:10.234723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:10.235481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:10.235769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:10.236176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] PATH_INFO: `/v3/projects/1a79b73c05184bb6b3586a71e6d2b31f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:10.237403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a79b73c05184bb6b3586a71e6d2b31f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:10.240339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:10.316904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48baba6e5a954fedbd024853e95eb38a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.344604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a79b73c05184bb6b3586a71e6d2b31f', 'request_id': 'req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.347837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48baba6e5a954fedbd024853e95eb38a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.349224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48baba6e5a954fedbd024853e95eb38a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.349598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48baba6e5a954fedbd024853e95eb38a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.365374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a79b73c05184bb6b3586a71e6d2b31f', 'request_id': 'req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.375872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48baba6e5a954fedbd024853e95eb38a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.378611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a79b73c05184bb6b3586a71e6d2b31f', 'request_id': 'req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.394396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] The token cache is being invalidate because project 1a79b73c05184bb6b3586a71e6d2b31f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:10.394702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.395412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bb4848b-826c-43c0-9bc5-b5ed1c81a88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.397247 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 829/1678] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:10 2022] DELETE /identity/v3/projects/1a79b73c05184bb6b3586a71e6d2b31f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:10.400830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc593867-8019-4974-bd76-4a948b846bec None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:10.400830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc593867-8019-4974-bd76-4a948b846bec None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:10.400830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc593867-8019-4974-bd76-4a948b846bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:10.438329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fc593867-8019-4974-bd76-4a948b846bec None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:10.440063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fc593867-8019-4974-bd76-4a948b846bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.445733 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 850/1679] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:10 2022] DELETE /identity/v3/users/48baba6e5a954fedbd024853e95eb38a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:10.448838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:10.448838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.535744 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 830/1680] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:10.535744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbd916-b594-49fb-bbb0-8dddf5239e62 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:10.535744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbd916-b594-49fb-bbb0-8dddf5239e62 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:10.535744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbd916-b594-49fb-bbb0-8dddf5239e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:10.567781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-14dbd916-b594-49fb-bbb0-8dddf5239e62 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:10.569541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-14dbd916-b594-49fb-bbb0-8dddf5239e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.588703 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 831/1681] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:10.588703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbd916-b594-49fb-bbb0-8dddf5239e62 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:10.588703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-14dbd916-b594-49fb-bbb0-8dddf5239e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.613713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbd916-b594-49fb-bbb0-8dddf5239e62 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:10.616677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebff1dc-7e4b-4dca-b61a-9d9e6451529d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:10.619363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:10.620079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:10.620422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:10.620706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] PATH_INFO: `/v3/users/a04a0db0efeb4de887c99deaa7124e6b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:10.622627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a04a0db0efeb4de887c99deaa7124e6b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:10.623179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1d89c-aa98-4cd6-b1d3-b0a342f53fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:10.623993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1d89c-aa98-4cd6-b1d3-b0a342f53fe8 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:10.624421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1d89c-aa98-4cd6-b1d3-b0a342f53fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:10.624779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1d89c-aa98-4cd6-b1d3-b0a342f53fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:10.625155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:10.626812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-68a1d89c-aa98-4cd6-b1d3-b0a342f53fe8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.671451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:10.743401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a04a0db0efeb4de887c99deaa7124e6b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.745552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1d89c-aa98-4cd6-b1d3-b0a342f53fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:10.749691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1d89c-aa98-4cd6-b1d3-b0a342f53fe8 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:10.752809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a04a0db0efeb4de887c99deaa7124e6b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.755594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a04a0db0efeb4de887c99deaa7124e6b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.757005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a04a0db0efeb4de887c99deaa7124e6b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.759372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:10.781177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a04a0db0efeb4de887c99deaa7124e6b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:10.786529 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 851/1682] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:10.790403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af12e-53f6-4d9b-b91e-ef611d8d5754 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:10.790719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af12e-53f6-4d9b-b91e-ef611d8d5754 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:10.791006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af12e-53f6-4d9b-b91e-ef611d8d5754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:10.823079 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 832/1683] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:10 2022] DELETE /identity/v3/users/a04a0db0efeb4de887c99deaa7124e6b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:10.823079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:10.825783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.864460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b76af12e-53f6-4d9b-b91e-ef611d8d5754 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:10.865907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b76af12e-53f6-4d9b-b91e-ef611d8d5754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:10.986545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e933b3-05bc-4525-a417-8197158546c4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:11.004096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb601189-3662-4ff6-82f4-0de8ef9f5b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:11.004779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb601189-3662-4ff6-82f4-0de8ef9f5b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.005152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb601189-3662-4ff6-82f4-0de8ef9f5b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.005520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb601189-3662-4ff6-82f4-0de8ef9f5b07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.009556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb601189-3662-4ff6-82f4-0de8ef9f5b07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:11.010594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb601189-3662-4ff6-82f4-0de8ef9f5b07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:11.017024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb601189-3662-4ff6-82f4-0de8ef9f5b07 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:11.049196 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 833/1684] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:11.055672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25f1c7-da69-4cfb-8d76-f197b23356fc None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.056145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25f1c7-da69-4cfb-8d76-f197b23356fc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.056520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25f1c7-da69-4cfb-8d76-f197b23356fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.076262 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 852/1685] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:11.081060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5fd0b-709a-4346-b662-769ba4df305d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.081350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5fd0b-709a-4346-b662-769ba4df305d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.081884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5fd0b-709a-4346-b662-769ba4df305d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.131580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2ce5fd0b-709a-4346-b662-769ba4df305d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:11.131580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2ce5fd0b-709a-4346-b662-769ba4df305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:11.134038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4f25f1c7-da69-4cfb-8d76-f197b23356fc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:11.136659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4f25f1c7-da69-4cfb-8d76-f197b23356fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:11.145122 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 853/1686] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:11.152321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bf78e-d6ca-4a35-9783-8f47c27527a4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.152619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bf78e-d6ca-4a35-9783-8f47c27527a4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.152898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bf78e-d6ca-4a35-9783-8f47c27527a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.188796 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 834/1687] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:11.214300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25f1c7-da69-4cfb-8d76-f197b23356fc None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:11.240661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25f1c7-da69-4cfb-8d76-f197b23356fc None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:11.246507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:11.247578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.247929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.248268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.249019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:11.251657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:11.264029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e82bf78e-d6ca-4a35-9783-8f47c27527a4 None None] MFA Rules not processed for user `c75799eec4744849a52d589e19424d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:11.265332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e82bf78e-d6ca-4a35-9783-8f47c27527a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:11.323255 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 835/1688] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:11 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:11.324336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:11.326434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:11.487348 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 854/1689] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:11.487348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bf78e-d6ca-4a35-9783-8f47c27527a4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:11.507504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7c84a-c80a-4c37-bfbf-e94447acfe9f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:11.514507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bf78e-d6ca-4a35-9783-8f47c27527a4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:11.518424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b14af-6eaa-42ac-9f9b-c917e05b563b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:11.519024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b14af-6eaa-42ac-9f9b-c917e05b563b service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.519320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b14af-6eaa-42ac-9f9b-c917e05b563b service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.519655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b14af-6eaa-42ac-9f9b-c917e05b563b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.666634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:11.678537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:11.691156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:11.691786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.692059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.692329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.692985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:11.693290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:11.700985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:11.723051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b14af-6eaa-42ac-9f9b-c917e05b563b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:11.724783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b14af-6eaa-42ac-9f9b-c917e05b563b service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:11.732586 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 855/1690] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:11.734807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:11.739850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:11.780727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27db84f-09d2-4884-b50d-692c2c238ae2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:11.783056 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 836/1691] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:11.789810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b14af-6eaa-42ac-9f9b-c917e05b563b service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:11.800816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1705d9-9ee0-4eae-bd9e-6c1b4c862115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:11.801451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1705d9-9ee0-4eae-bd9e-6c1b4c862115 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.801751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1705d9-9ee0-4eae-bd9e-6c1b4c862115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.802028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1705d9-9ee0-4eae-bd9e-6c1b4c862115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.816891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b14af-6eaa-42ac-9f9b-c917e05b563b service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:11.832084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1705d9-9ee0-4eae-bd9e-6c1b4c862115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:11.834713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f439b0-fc7a-4b71-ba83-0df9bb338232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:11.835568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f439b0-fc7a-4b71-ba83-0df9bb338232 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.836007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f439b0-fc7a-4b71-ba83-0df9bb338232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.836204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1705d9-9ee0-4eae-bd9e-6c1b4c862115 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:11.836589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f439b0-fc7a-4b71-ba83-0df9bb338232 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.837382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f439b0-fc7a-4b71-ba83-0df9bb338232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:11.842768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f439b0-fc7a-4b71-ba83-0df9bb338232 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:11.868500 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 856/1692] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:11.870125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ceda-1c1f-4759-9aad-b411af7e9136 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.870411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ceda-1c1f-4759-9aad-b411af7e9136 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.870687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ceda-1c1f-4759-9aad-b411af7e9136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.888381 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 837/1693] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:11 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:11.890637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f439b0-fc7a-4b71-ba83-0df9bb338232 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:11.918993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f439b0-fc7a-4b71-ba83-0df9bb338232 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:11.930628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ef51ceda-1c1f-4759-9aad-b411af7e9136 None None] MFA Rules not processed for user `c75799eec4744849a52d589e19424d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:11.931854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ef51ceda-1c1f-4759-9aad-b411af7e9136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:11.940602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816faa5-f17e-486e-b266-3cf1d871602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:11.941509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816faa5-f17e-486e-b266-3cf1d871602f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:11.942636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816faa5-f17e-486e-b266-3cf1d871602f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:11.943078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816faa5-f17e-486e-b266-3cf1d871602f admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:11.945441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816faa5-f17e-486e-b266-3cf1d871602f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:11.970359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816faa5-f17e-486e-b266-3cf1d871602f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:12.037501 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 838/1694] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:11 2022] POST /identity/v3/projects => generated 403 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:12.038968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816faa5-f17e-486e-b266-3cf1d871602f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:12.083205 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 857/1695] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:12.084081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51ceda-1c1f-4759-9aad-b411af7e9136 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:12.123484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51ceda-1c1f-4759-9aad-b411af7e9136 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:12.198663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816faa5-f17e-486e-b266-3cf1d871602f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:12.246004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:12.250857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.251543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.252192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.260123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:12.265038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:12.297853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 839/1696] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:12.299192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:12.302532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:12.315942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d4e95-bf0c-43ca-a55c-9039796e3b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:12.316583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d4e95-bf0c-43ca-a55c-9039796e3b62 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.316860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d4e95-bf0c-43ca-a55c-9039796e3b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.317591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d4e95-bf0c-43ca-a55c-9039796e3b62 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.320266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d4e95-bf0c-43ca-a55c-9039796e3b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:12.322783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d4e95-bf0c-43ca-a55c-9039796e3b62 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:12.370418 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 858/1697] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:12 2022] POST /identity/v3/users => generated 302 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:12.371563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d4e95-bf0c-43ca-a55c-9039796e3b62 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:12.401471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d4e95-bf0c-43ca-a55c-9039796e3b62 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:12.420774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6485ee-cc8b-44b1-9345-fcf9b0037dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:12.421682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6485ee-cc8b-44b1-9345-fcf9b0037dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.425038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6485ee-cc8b-44b1-9345-fcf9b0037dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.425521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6485ee-cc8b-44b1-9345-fcf9b0037dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.426449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6485ee-cc8b-44b1-9345-fcf9b0037dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:12.430513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6485ee-cc8b-44b1-9345-fcf9b0037dfc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:12.457123 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 859/1698] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:12.461645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6485ee-cc8b-44b1-9345-fcf9b0037dfc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:12.484011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6485ee-cc8b-44b1-9345-fcf9b0037dfc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:12.492115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d3f9f-23f8-419f-af75-a99e1d86f37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:12.492845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d3f9f-23f8-419f-af75-a99e1d86f37b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.493125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d3f9f-23f8-419f-af75-a99e1d86f37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.493451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d3f9f-23f8-419f-af75-a99e1d86f37b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.494185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d3f9f-23f8-419f-af75-a99e1d86f37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:12.496900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d3f9f-23f8-419f-af75-a99e1d86f37b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:12.519805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87578481-d055-46bc-9bd4-bfb8adfbc9bd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:12.530064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce4922-dfd2-4749-b875-d579c1a148d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:12.530701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4922-dfd2-4749-b875-d579c1a148d6 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.530986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4922-dfd2-4749-b875-d579c1a148d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.531260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4922-dfd2-4749-b875-d579c1a148d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.551034 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 860/1699] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:12.552079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d3f9f-23f8-419f-af75-a99e1d86f37b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:12.577037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d3f9f-23f8-419f-af75-a99e1d86f37b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:12.586636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20c993-667a-40d8-8b2a-d3fd8da47011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:12.588792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20c993-667a-40d8-8b2a-d3fd8da47011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.589370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20c993-667a-40d8-8b2a-d3fd8da47011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.589660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20c993-667a-40d8-8b2a-d3fd8da47011 admin admin] PATH_INFO: `/v3/projects/224e853b2f0740cc838a60ae46d63a2f/users/248a6a25184e4429b93b247d216f4fe7/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.616443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20c993-667a-40d8-8b2a-d3fd8da47011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224e853b2f0740cc838a60ae46d63a2f, user_id=248a6a25184e4429b93b247d216f4fe7, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:12.619573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20c993-667a-40d8-8b2a-d3fd8da47011 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:12.643188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:12.668127 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 861/1700] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:12 2022] PUT /identity/v3/projects/224e853b2f0740cc838a60ae46d63a2f/users/248a6a25184e4429b93b247d216f4fe7/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:12.669165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20c993-667a-40d8-8b2a-d3fd8da47011 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:12.691179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce4922-dfd2-4749-b875-d579c1a148d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:12.692826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce4922-dfd2-4749-b875-d579c1a148d6 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:12.717894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:12.721592 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 840/1701] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:12.725262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eca36-6efb-4a7a-becc-7d0f8d3edb18 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.726339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eca36-6efb-4a7a-becc-7d0f8d3edb18 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.727771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eca36-6efb-4a7a-becc-7d0f8d3edb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.810475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d72eca36-6efb-4a7a-becc-7d0f8d3edb18 None None] MFA Rules not processed for user `248a6a25184e4429b93b247d216f4fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:12.815291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d72eca36-6efb-4a7a-becc-7d0f8d3edb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:12.831201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20c993-667a-40d8-8b2a-d3fd8da47011 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:12.858637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:12.860875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.862332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.863574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.865806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:12.870778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:12.905121 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 862/1702] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:12.905121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:12.910773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:12.967118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394e4e5-5504-44ed-871f-e18154146b14 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:12.984591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:12.985309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:12.985691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:12.986047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] PATH_INFO: `/v3/projects/a56dc2359b184f4c9ba8b34b38e8f293` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:12.992431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56dc2359b184f4c9ba8b34b38e8f293)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:12.993585 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 30 09:36:12.993585 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Sep 30 09:36:12.993585 dv-cinder-master-3-smb devstack@keystone.service[81371]: Sep 30 09:36:12.997076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:13.018736 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 841/1703] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:13.025007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ff2e0-0eea-40f9-af9f-0aa4fe62f055 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:13.025367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ff2e0-0eea-40f9-af9f-0aa4fe62f055 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:13.025736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ff2e0-0eea-40f9-af9f-0aa4fe62f055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:13.100681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0b9ff2e0-0eea-40f9-af9f-0aa4fe62f055 None None] MFA Rules not processed for user `8efb5824b0c74435b41f3d7911c20874`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:13.102471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0b9ff2e0-0eea-40f9-af9f-0aa4fe62f055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:13.198169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56dc2359b184f4c9ba8b34b38e8f293', 'request_id': 'req-e14f198e-2a5f-4f71-adfd-d76218137ca0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:13.217220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56dc2359b184f4c9ba8b34b38e8f293', 'request_id': 'req-e14f198e-2a5f-4f71-adfd-d76218137ca0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:13.247672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56dc2359b184f4c9ba8b34b38e8f293', 'request_id': 'req-e14f198e-2a5f-4f71-adfd-d76218137ca0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:13.283106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] The token cache is being invalidate because project a56dc2359b184f4c9ba8b34b38e8f293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:13.283506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:13.285366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:13.289405 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 863/1704] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:12 2022] DELETE /identity/v3/projects/a56dc2359b184f4c9ba8b34b38e8f293 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:13.292754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:13.295581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:13.316400 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 842/1705] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:13.317754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ff2e0-0eea-40f9-af9f-0aa4fe62f055 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:13.498776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ff2e0-0eea-40f9-af9f-0aa4fe62f055 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:13.511666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:13.512310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:13.514175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:13.514462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:13.515128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:13.517082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:13.519949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f198e-2a5f-4f71-adfd-d76218137ca0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:13.534797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:13.536290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:13.536997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:13.537285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:13.537570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] PATH_INFO: `/v3/projects/33594f3cce6d402ea7aaa031691203b4/users/f6933cbaa2fd479a8a8d875aff96b63d/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:13.584117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33594f3cce6d402ea7aaa031691203b4, user_id=f6933cbaa2fd479a8a8d875aff96b63d, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:13.588606 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 843/1706] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:13 2022] POST /identity/v3/projects => generated 373 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:13.588606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:13.590128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:13.591273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:13.635230 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 864/1707] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:13 2022] PUT /identity/v3/projects/33594f3cce6d402ea7aaa031691203b4/users/f6933cbaa2fd479a8a8d875aff96b63d/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:13.635230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:13.637433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:13.711136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:13.722276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3985b-e01f-46b4-974f-d5ec153b057f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:13.817306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303ebd6-ab29-411d-b768-685c9368c917 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:13.831525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:13.833361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:13.833707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:13.835448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] PATH_INFO: `/v3/users/8d8d82a3514f440e8d31c9cdc9877108` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:13.837234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8d82a3514f440e8d31c9cdc9877108)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:13.842220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:13.890048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:13.891937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:13.892662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:13.895067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] PATH_INFO: `/v3/projects/aea5ebf794614ae0aedabca7fc761b27` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:13.899090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea5ebf794614ae0aedabca7fc761b27)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:13.903697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:13.967914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8d82a3514f440e8d31c9cdc9877108'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:13.983887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8d82a3514f440e8d31c9cdc9877108'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:13.986462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8d82a3514f440e8d31c9cdc9877108'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:13.988348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8d82a3514f440e8d31c9cdc9877108'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:14.005012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8d82a3514f440e8d31c9cdc9877108'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:14.018557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea5ebf794614ae0aedabca7fc761b27', 'request_id': 'req-7a4da895-2638-43ea-b86b-da53e8588355'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:14.037621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea5ebf794614ae0aedabca7fc761b27', 'request_id': 'req-7a4da895-2638-43ea-b86b-da53e8588355'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:14.046193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea5ebf794614ae0aedabca7fc761b27', 'request_id': 'req-7a4da895-2638-43ea-b86b-da53e8588355'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:14.057543 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 865/1708] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:13 2022] DELETE /identity/v3/users/8d8d82a3514f440e8d31c9cdc9877108 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:14.057543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:14.060047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:14.091832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] The token cache is being invalidate because project aea5ebf794614ae0aedabca7fc761b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:14.092173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:14.094843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:14.104780 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 844/1709] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:13 2022] DELETE /identity/v3/projects/aea5ebf794614ae0aedabca7fc761b27 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:14.105794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:14.110656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:14.196855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:14.257496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cebb04-7988-4649-bd1c-92c002cdc5be admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:14.277342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80484157-9843-46f2-bdb3-041b704fda32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:14.278011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80484157-9843-46f2-bdb3-041b704fda32 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.278296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80484157-9843-46f2-bdb3-041b704fda32 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.278574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80484157-9843-46f2-bdb3-041b704fda32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.286630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-80484157-9843-46f2-bdb3-041b704fda32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:14.297454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4da895-2638-43ea-b86b-da53e8588355 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:14.391500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:14.434599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca57c2-fe12-462d-a824-0f36bae361d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:14.435335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca57c2-fe12-462d-a824-0f36bae361d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.435721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca57c2-fe12-462d-a824-0f36bae361d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.436095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca57c2-fe12-462d-a824-0f36bae361d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.437143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca57c2-fe12-462d-a824-0f36bae361d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:14.442085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca57c2-fe12-462d-a824-0f36bae361d5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:14.478398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80484157-9843-46f2-bdb3-041b704fda32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:14.480244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80484157-9843-46f2-bdb3-041b704fda32 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:14.489472 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 845/1710] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:14 2022] POST /identity/v3/users => generated 293 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:14.489472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d85c9f-f511-4c05-be37-20a3ef298bf8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.489981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d85c9f-f511-4c05-be37-20a3ef298bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.490392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d85c9f-f511-4c05-be37-20a3ef298bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.498781 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 866/1711] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:14.500169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294df9e-9ec7-4e58-90df-b54bfdac591e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.500458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294df9e-9ec7-4e58-90df-b54bfdac591e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.500737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294df9e-9ec7-4e58-90df-b54bfdac591e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.510754 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 867/1712] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 30 09:36:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:36:14.514302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15932fb-30bd-43f7-9ea1-cb582fab5115 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.517518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15932fb-30bd-43f7-9ea1-cb582fab5115 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.518967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15932fb-30bd-43f7-9ea1-cb582fab5115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.574357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-28d85c9f-f511-4c05-be37-20a3ef298bf8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:14.579975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-28d85c9f-f511-4c05-be37-20a3ef298bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:14.608021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c15932fb-30bd-43f7-9ea1-cb582fab5115 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:14.609274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c15932fb-30bd-43f7-9ea1-cb582fab5115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:14.616769 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 846/1713] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:14.617782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d85c9f-f511-4c05-be37-20a3ef298bf8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:14.646009 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 868/1714] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:14.648881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce0e8-a02e-4456-bda5-2d6cbd193d31 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.649325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce0e8-a02e-4456-bda5-2d6cbd193d31 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.650249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbce0e8-a02e-4456-bda5-2d6cbd193d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.673470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d85c9f-f511-4c05-be37-20a3ef298bf8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:14.697994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:14.698776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.698916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.699194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.703842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:14.705485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:14.737258 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 847/1715] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:14.738726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:14.743527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:14.791146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-afbce0e8-a02e-4456-bda5-2d6cbd193d31 None None] MFA Rules not processed for user `f41e236339784d169258f06f1090b7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:14.793116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-afbce0e8-a02e-4456-bda5-2d6cbd193d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:14.916411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e563700-3cae-4c06-a876-47723ae413c2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:14.924533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd98f6-69be-4cb9-b279-e55dd976b108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:14.926175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd98f6-69be-4cb9-b279-e55dd976b108 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.927812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd98f6-69be-4cb9-b279-e55dd976b108 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.931012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd98f6-69be-4cb9-b279-e55dd976b108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.955497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd98f6-69be-4cb9-b279-e55dd976b108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:14.958747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd98f6-69be-4cb9-b279-e55dd976b108 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:14.977025 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 869/1716] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:36:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:14.980013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbcbbf-d3d9-43d4-a416-a7d5449604c6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:14.980345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbcbbf-d3d9-43d4-a416-a7d5449604c6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:14.980555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbcbbf-d3d9-43d4-a416-a7d5449604c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:14.991971 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 848/1717] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:14.993099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd98f6-69be-4cb9-b279-e55dd976b108 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:15.016283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd98f6-69be-4cb9-b279-e55dd976b108 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:15.031372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:15.032180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.032537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.032851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] PATH_INFO: `/v3/projects/5945ebed866f429792d0c60118b19c3c/users/a9a46dd480324b2f997c43dbbc67ddaa/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.034557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-04dbcbbf-d3d9-43d4-a416-a7d5449604c6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:15.036933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-04dbcbbf-d3d9-43d4-a416-a7d5449604c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.067402 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 870/1718] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:15.070548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbcbbf-d3d9-43d4-a416-a7d5449604c6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:15.074987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-04dbcbbf-d3d9-43d4-a416-a7d5449604c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.085653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5945ebed866f429792d0c60118b19c3c, user_id=a9a46dd480324b2f997c43dbbc67ddaa, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:15.094190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:15.113344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbcbbf-d3d9-43d4-a416-a7d5449604c6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:15.120986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:15.123376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.123851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.124265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] PATH_INFO: `/v3/projects/a7018406563b46f48900b2fe70332e7c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.125923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7018406563b46f48900b2fe70332e7c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:15.128511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:15.134353 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 849/1719] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:14 2022] PUT /identity/v3/projects/5945ebed866f429792d0c60118b19c3c/users/a9a46dd480324b2f997c43dbbc67ddaa/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:15.135280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:15.143049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.201988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7018406563b46f48900b2fe70332e7c', 'request_id': 'req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:15.206897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:15.216047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7018406563b46f48900b2fe70332e7c', 'request_id': 'req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:15.227916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7018406563b46f48900b2fe70332e7c', 'request_id': 'req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:15.246866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] The token cache is being invalidate because project a7018406563b46f48900b2fe70332e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:15.247711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:15.249567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:15.252518 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 871/1720] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:15 2022] DELETE /identity/v3/projects/a7018406563b46f48900b2fe70332e7c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:15.255585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:15.259180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.300928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e19de6-1045-4a4a-a73b-98af8cf53a83 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:15.343025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:15.380240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49d71b-dafa-4bc1-8709-3bb6cd3d3d37 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:15.389833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:15.390659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.391374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.391721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.392451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:15.392806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:15.395503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:15.428117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7a71-2aa4-4ccb-9ccf-dc658a1ef06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:15.429664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7a71-2aa4-4ccb-9ccf-dc658a1ef06c service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.430068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7a71-2aa4-4ccb-9ccf-dc658a1ef06c service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.430366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7a71-2aa4-4ccb-9ccf-dc658a1ef06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.435679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d24f7a71-2aa4-4ccb-9ccf-dc658a1ef06c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.444281 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 872/1721] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:15.446454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:15.452152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.464969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f7a71-2aa4-4ccb-9ccf-dc658a1ef06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:15.467504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f7a71-2aa4-4ccb-9ccf-dc658a1ef06c service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:15.493500 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 850/1722] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:15.495619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647dc0b-0bae-4877-82a5-66b7f7a54458 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.495909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647dc0b-0bae-4877-82a5-66b7f7a54458 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.496184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647dc0b-0bae-4877-82a5-66b7f7a54458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.498803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f318c07-6ed0-4993-a127-dbe7a62b9caf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:15.514528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216af89-7d85-4af3-81d8-a7653a4d7230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:15.519943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216af89-7d85-4af3-81d8-a7653a4d7230 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.520153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216af89-7d85-4af3-81d8-a7653a4d7230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.521790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216af89-7d85-4af3-81d8-a7653a4d7230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.523895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216af89-7d85-4af3-81d8-a7653a4d7230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:15.530455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216af89-7d85-4af3-81d8-a7653a4d7230 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:15.548265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2647dc0b-0bae-4877-82a5-66b7f7a54458 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:15.552309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2647dc0b-0bae-4877-82a5-66b7f7a54458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.558161 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 873/1723] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:15.559060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216af89-7d85-4af3-81d8-a7653a4d7230 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:15.575942 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 851/1724] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:15.579311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00d98c-6172-4a7c-bbfb-3434a1b47729 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.579612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00d98c-6172-4a7c-bbfb-3434a1b47729 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.580600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00d98c-6172-4a7c-bbfb-3434a1b47729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.591204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216af89-7d85-4af3-81d8-a7653a4d7230 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:15.609782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c49482-bacd-48cc-becd-5bf079ec664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:15.611077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c49482-bacd-48cc-becd-5bf079ec664d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.611569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c49482-bacd-48cc-becd-5bf079ec664d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.612052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c49482-bacd-48cc-becd-5bf079ec664d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.612898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c49482-bacd-48cc-becd-5bf079ec664d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:15.618094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c49482-bacd-48cc-becd-5bf079ec664d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:15.633319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7f00d98c-6172-4a7c-bbfb-3434a1b47729 None None] MFA Rules not processed for user `e32778edb0e943f48e5f40453c97322b`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:15.637171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7f00d98c-6172-4a7c-bbfb-3434a1b47729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.693150 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 874/1725] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:15 2022] POST /identity/v3/projects => generated 357 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:15.697032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c49482-bacd-48cc-becd-5bf079ec664d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:15.786514 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 852/1726] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:15.789373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71acfc5-d8fe-40b2-9c9c-ab125e70d924 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:15.789775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71acfc5-d8fe-40b2-9c9c-ab125e70d924 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:15.790054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71acfc5-d8fe-40b2-9c9c-ab125e70d924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:15.839585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c71acfc5-d8fe-40b2-9c9c-ab125e70d924 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:15.841162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c71acfc5-d8fe-40b2-9c9c-ab125e70d924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:15.988431 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 853/1727] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:15.990759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c49482-bacd-48cc-becd-5bf079ec664d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:15.990855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71acfc5-d8fe-40b2-9c9c-ab125e70d924 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:16.003977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be8a1e-a313-4384-9b44-1854f94e21d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:16.006149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be8a1e-a313-4384-9b44-1854f94e21d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.008579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be8a1e-a313-4384-9b44-1854f94e21d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.011170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be8a1e-a313-4384-9b44-1854f94e21d8 admin admin] PATH_INFO: `/v3/projects/5945ebed866f429792d0c60118b19c3c/users/a9a46dd480324b2f997c43dbbc67ddaa/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.022027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71acfc5-d8fe-40b2-9c9c-ab125e70d924 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:16.031970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be8a1e-a313-4384-9b44-1854f94e21d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5945ebed866f429792d0c60118b19c3c, user_id=a9a46dd480324b2f997c43dbbc67ddaa, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:16.034176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfe556-406f-438c-9547-ed7a4652bc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:16.034811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfe556-406f-438c-9547-ed7a4652bc43 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.035233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfe556-406f-438c-9547-ed7a4652bc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.035515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfe556-406f-438c-9547-ed7a4652bc43 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.036161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfe556-406f-438c-9547-ed7a4652bc43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:16.040498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfe556-406f-438c-9547-ed7a4652bc43 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:16.041462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be8a1e-a313-4384-9b44-1854f94e21d8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:16.134724 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 875/1728] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:15 2022] PUT /identity/v3/projects/5945ebed866f429792d0c60118b19c3c/users/a9a46dd480324b2f997c43dbbc67ddaa/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:16.134724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be8a1e-a313-4384-9b44-1854f94e21d8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:16.135885 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 854/1729] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:15 2022] POST /identity/v3/users => generated 279 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:16.140183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af482d7-19dc-4736-84cd-535e0aef82fe None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.140463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af482d7-19dc-4736-84cd-535e0aef82fe None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.140746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af482d7-19dc-4736-84cd-535e0aef82fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.172363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:16.278874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5af482d7-19dc-4736-84cd-535e0aef82fe None None] MFA Rules not processed for user `a9a46dd480324b2f997c43dbbc67ddaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:16.280153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5af482d7-19dc-4736-84cd-535e0aef82fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:16.383323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be8a1e-a313-4384-9b44-1854f94e21d8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:16.400323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:16.401294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.408679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.409293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.410162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:16.411544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:16.420468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:16.444918 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 876/1730] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:16.458225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:16.487574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ec3b4-6723-48fe-926e-73ce6202835a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:16.500417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:16.505109 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 855/1731] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:16.505109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af482d7-19dc-4736-84cd-535e0aef82fe None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:16.505334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.505334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.505334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.505334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:16.508511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:16.527232 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 877/1732] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:16.528328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:16.546571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:16.575355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af482d7-19dc-4736-84cd-535e0aef82fe None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:16.600191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbccac3-66fc-4a5a-8a63-35b7bab3e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:16.600191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbccac3-66fc-4a5a-8a63-35b7bab3e7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.600191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbccac3-66fc-4a5a-8a63-35b7bab3e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.600191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbccac3-66fc-4a5a-8a63-35b7bab3e7ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.601026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbccac3-66fc-4a5a-8a63-35b7bab3e7ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:16.609551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbccac3-66fc-4a5a-8a63-35b7bab3e7ca admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:16.625985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:16.720243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ec6ea-0a85-476d-b2dd-e902d4d01a9a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:16.728065 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 856/1733] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:16 2022] POST /identity/v3/projects => generated 369 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:16.736753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbccac3-66fc-4a5a-8a63-35b7bab3e7ca admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:16.853161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:16.855552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.857537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.859130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.886484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbccac3-66fc-4a5a-8a63-35b7bab3e7ca admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:16.914732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531635cb-ab2a-4590-bf8d-24468bef8b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:16.915885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531635cb-ab2a-4590-bf8d-24468bef8b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.916460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531635cb-ab2a-4590-bf8d-24468bef8b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.916892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531635cb-ab2a-4590-bf8d-24468bef8b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.917837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531635cb-ab2a-4590-bf8d-24468bef8b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:16.919734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531635cb-ab2a-4590-bf8d-24468bef8b0d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:16.943065 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 857/1734] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:16.958452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531635cb-ab2a-4590-bf8d-24468bef8b0d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:16.980353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531635cb-ab2a-4590-bf8d-24468bef8b0d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:16.993157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fadfd-1c94-404b-99b1-59e0807feb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:16.994509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fadfd-1c94-404b-99b1-59e0807feb5a admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:16.994755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fadfd-1c94-404b-99b1-59e0807feb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:16.995040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fadfd-1c94-404b-99b1-59e0807feb5a admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:16.995752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fadfd-1c94-404b-99b1-59e0807feb5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:16.998737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fadfd-1c94-404b-99b1-59e0807feb5a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:17.017994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:17.020199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:17.040573 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 878/1735] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:17.042191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:17.044137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:17.065448 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 858/1736] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:16 2022] POST /identity/v3/users => generated 285 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:17.066331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fadfd-1c94-404b-99b1-59e0807feb5a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:17.095898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fadfd-1c94-404b-99b1-59e0807feb5a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:17.110627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db610-e717-41fe-a11b-78feb7499227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:17.111449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db610-e717-41fe-a11b-78feb7499227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.112053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db610-e717-41fe-a11b-78feb7499227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.112328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db610-e717-41fe-a11b-78feb7499227 admin admin] PATH_INFO: `/v3/projects/7c2d8e595a7d4edba9d798b892dfc32f/users/f8d89ae5fffd470eaed7a3dcb09a3bd7/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.135014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31db610-e717-41fe-a11b-78feb7499227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2d8e595a7d4edba9d798b892dfc32f, user_id=f8d89ae5fffd470eaed7a3dcb09a3bd7, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:17.140203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31db610-e717-41fe-a11b-78feb7499227 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:17.165891 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 859/1737] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:17 2022] PUT /identity/v3/projects/7c2d8e595a7d4edba9d798b892dfc32f/users/f8d89ae5fffd470eaed7a3dcb09a3bd7/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:17.167416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db610-e717-41fe-a11b-78feb7499227 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:17.205495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:17.216683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3686ef8-b227-4ea5-b2c0-71eec8d039d7 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:17.281186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:17.318839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db610-e717-41fe-a11b-78feb7499227 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:17.332108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9bda4-cdc5-47d2-8dde-1a4d0343ace2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:17.332851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bda4-cdc5-47d2-8dde-1a4d0343ace2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.333266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bda4-cdc5-47d2-8dde-1a4d0343ace2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.333466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9bda4-cdc5-47d2-8dde-1a4d0343ace2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.336837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:17.343782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.343782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.343782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.344785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9bda4-cdc5-47d2-8dde-1a4d0343ace2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:17.355716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9bda4-cdc5-47d2-8dde-1a4d0343ace2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:17.361252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:17.400581 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 860/1738] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:17.400581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facca43-6d4a-431f-a315-c6617c82b3b5 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.401427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facca43-6d4a-431f-a315-c6617c82b3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.401738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facca43-6d4a-431f-a315-c6617c82b3b5 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.404391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:17.407083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:17.407662 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 861/1739] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 30 09:36:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:36:17.408792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f817e-25f6-4d77-9774-967cab4c5423 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.409077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f817e-25f6-4d77-9774-967cab4c5423 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.409350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f817e-25f6-4d77-9774-967cab4c5423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.448170 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 879/1740] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:17.448760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:17.504803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e89f817e-25f6-4d77-9774-967cab4c5423 None None] MFA Rules not processed for user `f8d89ae5fffd470eaed7a3dcb09a3bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:17.511578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e89f817e-25f6-4d77-9774-967cab4c5423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:17.648995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f622f-e21c-4b7a-89dc-3af66335fa38 service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:17.676674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff928c-6471-4977-9e13-5790dd47c7fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:17.677452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff928c-6471-4977-9e13-5790dd47c7fc service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.679721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff928c-6471-4977-9e13-5790dd47c7fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.681347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff928c-6471-4977-9e13-5790dd47c7fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.687179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-07ff928c-6471-4977-9e13-5790dd47c7fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:17.784331 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 862/1741] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:17.785809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f817e-25f6-4d77-9774-967cab4c5423 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:17.825603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f817e-25f6-4d77-9774-967cab4c5423 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:17.836267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f074f6a-8317-43b7-9c02-2902b8b93bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:17.836985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f074f6a-8317-43b7-9c02-2902b8b93bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.837264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f074f6a-8317-43b7-9c02-2902b8b93bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.837534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f074f6a-8317-43b7-9c02-2902b8b93bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.838404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f074f6a-8317-43b7-9c02-2902b8b93bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:17.842375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f074f6a-8317-43b7-9c02-2902b8b93bb0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:17.862485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff928c-6471-4977-9e13-5790dd47c7fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:17.866676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff928c-6471-4977-9e13-5790dd47c7fc service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:17.877483 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 863/1742] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:17.878824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d8eb-1854-4aa7-a905-61c776e4a020 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.879374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d8eb-1854-4aa7-a905-61c776e4a020 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.879640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d8eb-1854-4aa7-a905-61c776e4a020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.900270 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 880/1743] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:17.900270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2730cc8-8d69-4c0c-9f8c-3421c16b0b16 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:17.900270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2730cc8-8d69-4c0c-9f8c-3421c16b0b16 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:17.900270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2730cc8-8d69-4c0c-9f8c-3421c16b0b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:17.940917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9765d8eb-1854-4aa7-a905-61c776e4a020 None None] MFA Rules not processed for user `f41e236339784d169258f06f1090b7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:17.942254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9765d8eb-1854-4aa7-a905-61c776e4a020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:17.970891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a2730cc8-8d69-4c0c-9f8c-3421c16b0b16 None None] MFA Rules not processed for user `f6933cbaa2fd479a8a8d875aff96b63d`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:17.974149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a2730cc8-8d69-4c0c-9f8c-3421c16b0b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:18.101950 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 864/1744] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:18.106899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765d8eb-1854-4aa7-a905-61c776e4a020 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:18.140784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765d8eb-1854-4aa7-a905-61c776e4a020 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:18.156352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:18.157153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:18.157986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:18.157986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:18.160278 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 881/1745] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:18.161197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2730cc8-8d69-4c0c-9f8c-3421c16b0b16 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:18.162860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:18.165618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:18.188797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:18.200502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2730cc8-8d69-4c0c-9f8c-3421c16b0b16 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:18.224237 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 865/1746] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:18 2022] POST /identity/v3/projects => generated 357 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:18.225993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:18.231186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:18.237716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6107796-7eb2-468d-ad71-42aa5c923def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:18.246527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6107796-7eb2-468d-ad71-42aa5c923def admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:18.246823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6107796-7eb2-468d-ad71-42aa5c923def admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:18.247119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6107796-7eb2-468d-ad71-42aa5c923def admin admin] PATH_INFO: `/v3/projects/4c66a4ab9bf9477ca5236dedb67c4617/users/2d32d61be57c40e488840c9b8a53de7d/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:18.309283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6107796-7eb2-468d-ad71-42aa5c923def admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c66a4ab9bf9477ca5236dedb67c4617, user_id=2d32d61be57c40e488840c9b8a53de7d, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:18.315958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6107796-7eb2-468d-ad71-42aa5c923def admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:18.355519 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 882/1747] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:18 2022] PUT /identity/v3/projects/4c66a4ab9bf9477ca5236dedb67c4617/users/2d32d61be57c40e488840c9b8a53de7d/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:18.355519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6107796-7eb2-468d-ad71-42aa5c923def admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:18.385197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:18.504597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97b55a-18c2-4783-9a5c-89f7ce4f4ccb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:18.537235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6107796-7eb2-468d-ad71-42aa5c923def admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:18.547564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:18.548195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:18.548585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:18.548747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:18.549413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:18.551442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:18.598994 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 883/1748] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:18 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:18.599971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:18.603640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:18.736108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422760c7-2056-401c-86da-bd40e027d057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:18.736784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422760c7-2056-401c-86da-bd40e027d057 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:18.737043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422760c7-2056-401c-86da-bd40e027d057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:18.737306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422760c7-2056-401c-86da-bd40e027d057 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:18.738043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422760c7-2056-401c-86da-bd40e027d057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:18.739981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422760c7-2056-401c-86da-bd40e027d057 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:18.783412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a532f84-7bee-41a9-8190-fdfff0fe72e9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:18.792269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30b441-6ba1-4713-8088-fdbcd4857588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:18.793986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30b441-6ba1-4713-8088-fdbcd4857588 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:18.794490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30b441-6ba1-4713-8088-fdbcd4857588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:18.795016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30b441-6ba1-4713-8088-fdbcd4857588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:18.811490 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 866/1749] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:18 2022] POST /identity/v3/projects => generated 403 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:18.812220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30b441-6ba1-4713-8088-fdbcd4857588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:18.813906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30b441-6ba1-4713-8088-fdbcd4857588 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:18.814468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e36a6-a81a-41c5-9245-3dc899056124 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:18.814856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e36a6-a81a-41c5-9245-3dc899056124 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:18.815244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e36a6-a81a-41c5-9245-3dc899056124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:18.825480 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 884/1750] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:18.826738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30b441-6ba1-4713-8088-fdbcd4857588 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:18.850635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8e4e36a6-a81a-41c5-9245-3dc899056124 None None] MFA Rules not processed for user `2d32d61be57c40e488840c9b8a53de7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:18.852472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e4e36a6-a81a-41c5-9245-3dc899056124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:18.855182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30b441-6ba1-4713-8088-fdbcd4857588 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:18.951439 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 867/1751] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:18.953111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36138-1714-4e57-8a5b-8e2cdf5435b7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:18.954806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36138-1714-4e57-8a5b-8e2cdf5435b7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:18.955192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd36138-1714-4e57-8a5b-8e2cdf5435b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:18.962071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e13ece-9a34-41a5-ae0d-824909d0452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:18.963775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e13ece-9a34-41a5-ae0d-824909d0452c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:18.964213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e13ece-9a34-41a5-ae0d-824909d0452c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:18.964647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e13ece-9a34-41a5-ae0d-824909d0452c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:18.965521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e13ece-9a34-41a5-ae0d-824909d0452c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:18.968280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e13ece-9a34-41a5-ae0d-824909d0452c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:18.980226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5dd36138-1714-4e57-8a5b-8e2cdf5435b7 None None] MFA Rules not processed for user `8efb5824b0c74435b41f3d7911c20874`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:18.982267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5dd36138-1714-4e57-8a5b-8e2cdf5435b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:18.986068 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 885/1752] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:18.987509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e13ece-9a34-41a5-ae0d-824909d0452c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.000526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e13ece-9a34-41a5-ae0d-824909d0452c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.013325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434cc40-8097-4ebc-b8b5-34e440181b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.014370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434cc40-8097-4ebc-b8b5-34e440181b4f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.014819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434cc40-8097-4ebc-b8b5-34e440181b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.015367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434cc40-8097-4ebc-b8b5-34e440181b4f admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.017255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434cc40-8097-4ebc-b8b5-34e440181b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.023905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434cc40-8097-4ebc-b8b5-34e440181b4f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.039250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:19.043358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:19.076486 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 886/1753] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:18 2022] POST /identity/v3/users => generated 308 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:19.078521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434cc40-8097-4ebc-b8b5-34e440181b4f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.092183 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 868/1754] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:19.093653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd36138-1714-4e57-8a5b-8e2cdf5435b7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.118417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434cc40-8097-4ebc-b8b5-34e440181b4f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.139526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41df4b7-b674-43d3-97c3-c235bfae64b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.140163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df4b7-b674-43d3-97c3-c235bfae64b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.140439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df4b7-b674-43d3-97c3-c235bfae64b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.140708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df4b7-b674-43d3-97c3-c235bfae64b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.142086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41df4b7-b674-43d3-97c3-c235bfae64b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.144726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41df4b7-b674-43d3-97c3-c235bfae64b6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.159237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd36138-1714-4e57-8a5b-8e2cdf5435b7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.172376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ee095-2d6d-47e1-b04c-3a85476b8263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.174141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee095-2d6d-47e1-b04c-3a85476b8263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.174372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee095-2d6d-47e1-b04c-3a85476b8263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.175270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee095-2d6d-47e1-b04c-3a85476b8263 admin admin] PATH_INFO: `/v3/projects/574f5dfe9f4344bfabe064c2367098cd/users/f131966abe52457a9a64ae0faba9b628/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.203901 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 887/1755] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:19 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:19.209102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41df4b7-b674-43d3-97c3-c235bfae64b6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.226666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ee095-2d6d-47e1-b04c-3a85476b8263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574f5dfe9f4344bfabe064c2367098cd, user_id=f131966abe52457a9a64ae0faba9b628, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.229486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ee095-2d6d-47e1-b04c-3a85476b8263 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.287640 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 869/1756] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:19 2022] PUT /identity/v3/projects/574f5dfe9f4344bfabe064c2367098cd/users/f131966abe52457a9a64ae0faba9b628/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:19.296660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ee095-2d6d-47e1-b04c-3a85476b8263 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.406246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41df4b7-b674-43d3-97c3-c235bfae64b6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.485394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ee095-2d6d-47e1-b04c-3a85476b8263 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.491883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae8a5e-eeb8-4298-96b0-b20c79931349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.492505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae8a5e-eeb8-4298-96b0-b20c79931349 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.492772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae8a5e-eeb8-4298-96b0-b20c79931349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.493036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae8a5e-eeb8-4298-96b0-b20c79931349 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.493701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae8a5e-eeb8-4298-96b0-b20c79931349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.495878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae8a5e-eeb8-4298-96b0-b20c79931349 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.526961 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 870/1757] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:19 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:19.529051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae8a5e-eeb8-4298-96b0-b20c79931349 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.542085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae8a5e-eeb8-4298-96b0-b20c79931349 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.551052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ae5b0-787e-4c7e-8f25-7813c8063ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.556731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ae5b0-787e-4c7e-8f25-7813c8063ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.558251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ae5b0-787e-4c7e-8f25-7813c8063ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.558843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ae5b0-787e-4c7e-8f25-7813c8063ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.559820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ae5b0-787e-4c7e-8f25-7813c8063ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.565443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ae5b0-787e-4c7e-8f25-7813c8063ddc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.599025 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 871/1758] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:19.602080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc6ff3-ff2b-4040-862c-1e96447196be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.604136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc6ff3-ff2b-4040-862c-1e96447196be admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.604333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ae5b0-787e-4c7e-8f25-7813c8063ddc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.605805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc6ff3-ff2b-4040-862c-1e96447196be admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.606234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc6ff3-ff2b-4040-862c-1e96447196be admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.609487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc6ff3-ff2b-4040-862c-1e96447196be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.614597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc6ff3-ff2b-4040-862c-1e96447196be admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.628857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ae5b0-787e-4c7e-8f25-7813c8063ddc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.642319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d31ff-5049-4a0a-ae20-20e06ee272f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.644893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d31ff-5049-4a0a-ae20-20e06ee272f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.645334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d31ff-5049-4a0a-ae20-20e06ee272f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.645779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d31ff-5049-4a0a-ae20-20e06ee272f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.648770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d31ff-5049-4a0a-ae20-20e06ee272f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.652184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d31ff-5049-4a0a-ae20-20e06ee272f1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.668246 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 888/1759] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:19.669423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc6ff3-ff2b-4040-862c-1e96447196be admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.679519 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 872/1760] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:19.680503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d31ff-5049-4a0a-ae20-20e06ee272f1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.705903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc6ff3-ff2b-4040-862c-1e96447196be admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.710097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d31ff-5049-4a0a-ae20-20e06ee272f1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:19.723752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec63e0-c289-477b-9ca5-b4a67e908f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.724917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec63e0-c289-477b-9ca5-b4a67e908f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.725238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec63e0-c289-477b-9ca5-b4a67e908f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.729806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec63e0-c289-477b-9ca5-b4a67e908f85 admin admin] PATH_INFO: `/v3/projects/574f5dfe9f4344bfabe064c2367098cd/users/f131966abe52457a9a64ae0faba9b628/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.729982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceba2a2-a4b8-4fa5-b5ea-7a1a7fbcf6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:19.730593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceba2a2-a4b8-4fa5-b5ea-7a1a7fbcf6de admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:19.731037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceba2a2-a4b8-4fa5-b5ea-7a1a7fbcf6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:19.731323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceba2a2-a4b8-4fa5-b5ea-7a1a7fbcf6de admin admin] PATH_INFO: `/v3/projects/0d563afa0afd4dae894d8256e838aeb5/users/aa754aeb51b54bbb9c5fc636a7fcb72a/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:19.736523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec63e0-c289-477b-9ca5-b4a67e908f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574f5dfe9f4344bfabe064c2367098cd, user_id=f131966abe52457a9a64ae0faba9b628, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.761402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceba2a2-a4b8-4fa5-b5ea-7a1a7fbcf6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d563afa0afd4dae894d8256e838aeb5, user_id=aa754aeb51b54bbb9c5fc636a7fcb72a, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:19.764651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceba2a2-a4b8-4fa5-b5ea-7a1a7fbcf6de admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.771917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec63e0-c289-477b-9ca5-b4a67e908f85 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:19.784481 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 873/1761] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:19 2022] PUT /identity/v3/projects/0d563afa0afd4dae894d8256e838aeb5/users/aa754aeb51b54bbb9c5fc636a7fcb72a/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:19.785746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceba2a2-a4b8-4fa5-b5ea-7a1a7fbcf6de admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:19.841799 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 889/1762] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:19 2022] PUT /identity/v3/projects/574f5dfe9f4344bfabe064c2367098cd/users/f131966abe52457a9a64ae0faba9b628/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:19.844791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec63e0-c289-477b-9ca5-b4a67e908f85 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:20.103004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec63e0-c289-477b-9ca5-b4a67e908f85 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:20.111541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6254c-d220-41b1-bec3-dcf059298687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:20.112227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6254c-d220-41b1-bec3-dcf059298687 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:20.112584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6254c-d220-41b1-bec3-dcf059298687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:20.112893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6254c-d220-41b1-bec3-dcf059298687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:20.113562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6254c-d220-41b1-bec3-dcf059298687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:20.117500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6254c-d220-41b1-bec3-dcf059298687 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:20.119670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:20.138801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceba2a2-a4b8-4fa5-b5ea-7a1a7fbcf6de admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:20.140398 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 890/1763] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:20.141925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27eb03-ba19-4ef3-91b9-db316af7706a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:20.142213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27eb03-ba19-4ef3-91b9-db316af7706a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:20.142487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27eb03-ba19-4ef3-91b9-db316af7706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:20.157223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd583a-deab-401b-a6d4-776af98cf8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:20.158473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd583a-deab-401b-a6d4-776af98cf8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:20.160513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd583a-deab-401b-a6d4-776af98cf8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:20.162310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd583a-deab-401b-a6d4-776af98cf8c9 admin admin] PATH_INFO: `/v3/projects/854c589c26594f7090a7e2ab73b8744f/users/6c023db0bd084ae2829ec7aa11a6dc4d/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:20.208961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:20.216143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd583a-deab-401b-a6d4-776af98cf8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854c589c26594f7090a7e2ab73b8744f, user_id=6c023db0bd084ae2829ec7aa11a6dc4d, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:20.223212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd583a-deab-401b-a6d4-776af98cf8c9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:20.230724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5b27eb03-ba19-4ef3-91b9-db316af7706a None None] MFA Rules not processed for user `f131966abe52457a9a64ae0faba9b628`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:20.231971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5b27eb03-ba19-4ef3-91b9-db316af7706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:20.258058 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 874/1764] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:19 2022] PUT /identity/v3/projects/854c589c26594f7090a7e2ab73b8744f/users/6c023db0bd084ae2829ec7aa11a6dc4d/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:20.258058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd583a-deab-401b-a6d4-776af98cf8c9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:20.457904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd583a-deab-401b-a6d4-776af98cf8c9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:20.480221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aef896-19ca-47e1-aff7-437e52f2c7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:20.480934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef896-19ca-47e1-aff7-437e52f2c7a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:20.481222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef896-19ca-47e1-aff7-437e52f2c7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:20.481505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aef896-19ca-47e1-aff7-437e52f2c7a3 admin admin] PATH_INFO: `/v3/projects/0d563afa0afd4dae894d8256e838aeb5/users/aa754aeb51b54bbb9c5fc636a7fcb72a/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:20.487485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aef896-19ca-47e1-aff7-437e52f2c7a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d563afa0afd4dae894d8256e838aeb5, user_id=aa754aeb51b54bbb9c5fc636a7fcb72a, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:20.490696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aef896-19ca-47e1-aff7-437e52f2c7a3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:20.527049 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 875/1765] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:20 2022] PUT /identity/v3/projects/0d563afa0afd4dae894d8256e838aeb5/users/aa754aeb51b54bbb9c5fc636a7fcb72a/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:20.527049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aef896-19ca-47e1-aff7-437e52f2c7a3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:20.559521 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 891/1766] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:20.559521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27eb03-ba19-4ef3-91b9-db316af7706a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:20.564057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5b27eb03-ba19-4ef3-91b9-db316af7706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:20.634571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27eb03-ba19-4ef3-91b9-db316af7706a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:20.706675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aef896-19ca-47e1-aff7-437e52f2c7a3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:20.723391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:20.724046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:20.724339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:20.724638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:20.725844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:20.732918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:20.773706 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 876/1767] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:20.774915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:20.780892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:20.794318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:20.799165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:20.799607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:20.800178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] PATH_INFO: `/v3/users/5b9c41ae78f941b7ad698693f29c4a3d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:20.801845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b9c41ae78f941b7ad698693f29c4a3d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:20.804860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:20.915777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b9c41ae78f941b7ad698693f29c4a3d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:20.929775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b9c41ae78f941b7ad698693f29c4a3d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:20.930498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b9c41ae78f941b7ad698693f29c4a3d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:20.931440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b9c41ae78f941b7ad698693f29c4a3d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:20.946650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be739a-ad41-40b4-b3c0-c471015624aa admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:20.951033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b9c41ae78f941b7ad698693f29c4a3d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:20.956897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d047fc53-b7b9-4662-b0f4-b9036fa7087a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:20.960034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047fc53-b7b9-4662-b0f4-b9036fa7087a service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:20.960701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047fc53-b7b9-4662-b0f4-b9036fa7087a service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:20.960701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047fc53-b7b9-4662-b0f4-b9036fa7087a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:20.976793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d047fc53-b7b9-4662-b0f4-b9036fa7087a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:21.012661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:21.035053 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 892/1768] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:20 2022] DELETE /identity/v3/users/5b9c41ae78f941b7ad698693f29c4a3d => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:21.035053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:21.041132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:21.161649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:21.254580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047fc53-b7b9-4662-b0f4-b9036fa7087a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:21.258958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047fc53-b7b9-4662-b0f4-b9036fa7087a service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:21.282928 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 877/1769] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:21.285060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9125cc1-5c86-44e8-962f-cb70f2b067f2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:21.285930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9125cc1-5c86-44e8-962f-cb70f2b067f2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:21.287131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9125cc1-5c86-44e8-962f-cb70f2b067f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:21.342194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c9125cc1-5c86-44e8-962f-cb70f2b067f2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:21.345951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90095f-129a-418b-a2b1-53a61f3361b7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:21.347732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c9125cc1-5c86-44e8-962f-cb70f2b067f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:21.377053 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 878/1770] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:21.383710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768e4f7-cdb7-4062-94ce-9eb5018ff2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:21.386961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768e4f7-cdb7-4062-94ce-9eb5018ff2da admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:21.389467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768e4f7-cdb7-4062-94ce-9eb5018ff2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:21.389925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768e4f7-cdb7-4062-94ce-9eb5018ff2da admin admin] PATH_INFO: `/v3/projects/854c589c26594f7090a7e2ab73b8744f/users/6c023db0bd084ae2829ec7aa11a6dc4d/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:21.392649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768e4f7-cdb7-4062-94ce-9eb5018ff2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=854c589c26594f7090a7e2ab73b8744f, user_id=6c023db0bd084ae2829ec7aa11a6dc4d, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:21.399087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768e4f7-cdb7-4062-94ce-9eb5018ff2da admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:21.424556 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 893/1771] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:21 2022] PUT /identity/v3/projects/854c589c26594f7090a7e2ab73b8744f/users/6c023db0bd084ae2829ec7aa11a6dc4d/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:21.439464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4d1b-b9a2-4330-9921-bf48b163c087 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:21.439752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4d1b-b9a2-4330-9921-bf48b163c087 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:21.440030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4d1b-b9a2-4330-9921-bf48b163c087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:21.450103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873b20-1cc6-4a5a-831c-84f5b04b2cf0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:21.450419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873b20-1cc6-4a5a-831c-84f5b04b2cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:21.450662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873b20-1cc6-4a5a-831c-84f5b04b2cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:21.546681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-768e4d1b-b9a2-4330-9921-bf48b163c087 None None] MFA Rules not processed for user `6c023db0bd084ae2829ec7aa11a6dc4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:21.550810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-768e4d1b-b9a2-4330-9921-bf48b163c087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:21.685296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f1873b20-1cc6-4a5a-831c-84f5b04b2cf0 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:21.688969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f1873b20-1cc6-4a5a-831c-84f5b04b2cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:21.776459 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 879/1772] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:21.780854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e4d1b-b9a2-4330-9921-bf48b163c087 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:21.781839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-768e4d1b-b9a2-4330-9921-bf48b163c087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:21.909407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e4d1b-b9a2-4330-9921-bf48b163c087 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:21.918738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:21.919378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:21.919662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:21.919941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:21.930953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:21.962684 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 894/1773] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:21.964253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873b20-1cc6-4a5a-831c-84f5b04b2cf0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:21.966404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f1873b20-1cc6-4a5a-831c-84f5b04b2cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:22.076439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:22.078119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:22.102143 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 880/1774] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:22.103102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:22.112125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:22.121726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:22.130334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184e0a-f282-4abd-97eb-2f4d36777b8c service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:22.140395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:22.142667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:22.142667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:22.142667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:22.165492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:22.167405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:22.196106 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 881/1775] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:22.226264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873b20-1cc6-4a5a-831c-84f5b04b2cf0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:22.236505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba5cbd-3a00-48cf-bcb1-59babd54a561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:22.237782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba5cbd-3a00-48cf-bcb1-59babd54a561 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:22.238074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba5cbd-3a00-48cf-bcb1-59babd54a561 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:22.238707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba5cbd-3a00-48cf-bcb1-59babd54a561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:22.281272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:22.282764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:22.314942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:22.457580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba5cbd-3a00-48cf-bcb1-59babd54a561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:22.460137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba5cbd-3a00-48cf-bcb1-59babd54a561 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:22.495638 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 895/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:22.498771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba5cbd-3a00-48cf-bcb1-59babd54a561 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:22.525786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba5cbd-3a00-48cf-bcb1-59babd54a561 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:22.533978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:22.534786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:22.535828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:22.536119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:22.542431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:22.599153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee046382-b1c0-4c4d-b762-2227ac888994 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:22.614727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f5e83-7f21-47ff-ac3b-c2c5b8025d5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:22.616166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f5e83-7f21-47ff-ac3b-c2c5b8025d5e service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:22.616616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f5e83-7f21-47ff-ac3b-c2c5b8025d5e service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:22.617036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f5e83-7f21-47ff-ac3b-c2c5b8025d5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:22.660300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f5e83-7f21-47ff-ac3b-c2c5b8025d5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:22.664254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f5e83-7f21-47ff-ac3b-c2c5b8025d5e service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:22.684956 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 882/1777] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:36:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 30 09:36:22.846507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:22.848176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:22.871379 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 896/1778] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:23.020279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695cd33-83cb-4b60-bc22-e2aea4d022e4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:23.020563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695cd33-83cb-4b60-bc22-e2aea4d022e4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:23.021362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695cd33-83cb-4b60-bc22-e2aea4d022e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:23.079734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a695cd33-83cb-4b60-bc22-e2aea4d022e4 None None] MFA Rules not processed for user `aa754aeb51b54bbb9c5fc636a7fcb72a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:23.083484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a695cd33-83cb-4b60-bc22-e2aea4d022e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:23.290541 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 883/1779] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:24.662388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:24.665186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:24.774521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897f44-e4c9-442f-9821-f7d1c07ccd47 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:24.791194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:24.791643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:24.791939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:24.792221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] PATH_INFO: `/v3/projects/bb6c2450405e46c3b4b30cb58934e123` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:24.798437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6c2450405e46c3b4b30cb58934e123)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:24.804643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:24.987818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6c2450405e46c3b4b30cb58934e123', 'request_id': 'req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.011198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6c2450405e46c3b4b30cb58934e123', 'request_id': 'req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.051978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6c2450405e46c3b4b30cb58934e123', 'request_id': 'req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.124136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] The token cache is being invalidate because project bb6c2450405e46c3b4b30cb58934e123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:25.126433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.128627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b3c094b8-2aa6-4d5a-a2f1-c2b797cc1b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.138785 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 897/1780] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:24 2022] DELETE /identity/v3/projects/bb6c2450405e46c3b4b30cb58934e123 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:25.142853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695cd33-83cb-4b60-bc22-e2aea4d022e4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:25.166137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695cd33-83cb-4b60-bc22-e2aea4d022e4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:25.315170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:25.351221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:25.351947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:25.352405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:25.352832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] PATH_INFO: `/v3/users/0af61f60c0ea4725bcddbd311734946a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:25.354319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af61f60c0ea4725bcddbd311734946a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:25.356633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:25.492146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af61f60c0ea4725bcddbd311734946a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.503086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af61f60c0ea4725bcddbd311734946a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.504191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af61f60c0ea4725bcddbd311734946a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.504904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5d3de-a78e-46cb-80c1-aaacf7a18fa5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:25.505186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af61f60c0ea4725bcddbd311734946a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.506128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5d3de-a78e-46cb-80c1-aaacf7a18fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:25.506459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5d3de-a78e-46cb-80c1-aaacf7a18fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:25.533940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af61f60c0ea4725bcddbd311734946a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:25.574560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-65f5d3de-a78e-46cb-80c1-aaacf7a18fa5 None None] MFA Rules not processed for user `f41e236339784d169258f06f1090b7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:25.593551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-65f5d3de-a78e-46cb-80c1-aaacf7a18fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:25.604371 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 884/1781] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:25 2022] DELETE /identity/v3/users/0af61f60c0ea4725bcddbd311734946a => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:25.716778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:25.763392 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 898/1782] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Sep 30 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:25.852768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:25.854291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:25.888551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45707154-a811-4a4f-9d2a-340eb4c7b73a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:25.892559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7324c-743a-460c-9a27-847ac78dcbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:25.893238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7324c-743a-460c-9a27-847ac78dcbaf service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:25.893523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7324c-743a-460c-9a27-847ac78dcbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:25.893905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7324c-743a-460c-9a27-847ac78dcbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:25.919689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7324c-743a-460c-9a27-847ac78dcbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:25.921192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7324c-743a-460c-9a27-847ac78dcbaf service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:25.950915 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 885/1783] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:26.435300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab3a01-d96a-4105-8123-9c4a4726a376 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:26.435691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab3a01-d96a-4105-8123-9c4a4726a376 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:26.435865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab3a01-d96a-4105-8123-9c4a4726a376 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:26.438070 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 899/1784] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:36:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:36:26.471532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8b75-69e6-4eb9-b204-d6bef5de9670 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:26.472831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8b75-69e6-4eb9-b204-d6bef5de9670 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:26.473942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a8b75-69e6-4eb9-b204-d6bef5de9670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:26.516817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0d9a8b75-69e6-4eb9-b204-d6bef5de9670 None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:26.518302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0d9a8b75-69e6-4eb9-b204-d6bef5de9670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:26.671750 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 886/1785] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:36:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:26.785765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab3a01-d96a-4105-8123-9c4a4726a376 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:26.789200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1bab3a01-d96a-4105-8123-9c4a4726a376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:26.831515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab3a01-d96a-4105-8123-9c4a4726a376 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:26.847692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:26.848432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:26.848722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:26.848996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:26.891106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:26.893684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:26.922391 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 900/1786] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:28.063330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a8b75-69e6-4eb9-b204-d6bef5de9670 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:28.064981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0d9a8b75-69e6-4eb9-b204-d6bef5de9670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:28.180317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:28.216597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a8b75-69e6-4eb9-b204-d6bef5de9670 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:28.230167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:28.230963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:28.231448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:28.232717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] PATH_INFO: `/v3/users/f6933cbaa2fd479a8a8d875aff96b63d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:28.236275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6933cbaa2fd479a8a8d875aff96b63d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:28.239468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:28.425836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6933cbaa2fd479a8a8d875aff96b63d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:28.456239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6933cbaa2fd479a8a8d875aff96b63d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:28.459356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6933cbaa2fd479a8a8d875aff96b63d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:28.460830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6933cbaa2fd479a8a8d875aff96b63d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:28.486142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6933cbaa2fd479a8a8d875aff96b63d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:28.532071 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 887/1787] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:28 2022] DELETE /identity/v3/users/f6933cbaa2fd479a8a8d875aff96b63d => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:28.919150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:28.920853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:28.933117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:28.934961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:28.963992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dc98f-6f8a-4ca3-a6a3-d2a8ef7ff3b2 service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:29.092836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:29.093588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:29.094138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:29.094587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] PATH_INFO: `/v3/projects/8b05fd3ff10c4d66bdfb3446ab709604` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:29.096352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b05fd3ff10c4d66bdfb3446ab709604)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:29.098974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:29.185932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b8434-b143-455a-9f8e-3cb7da5f71ac admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:29.200678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b05fd3ff10c4d66bdfb3446ab709604', 'request_id': 'req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.215208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b05fd3ff10c4d66bdfb3446ab709604', 'request_id': 'req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.224645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b05fd3ff10c4d66bdfb3446ab709604', 'request_id': 'req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.242029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] The token cache is being invalidate because project 8b05fd3ff10c4d66bdfb3446ab709604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:29.243642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.244707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f322557-e1df-4ff6-ac2a-4d38fa3426b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.247393 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 901/1788] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:28 2022] DELETE /identity/v3/projects/8b05fd3ff10c4d66bdfb3446ab709604 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:29.252184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6f7e3-0936-4a6d-a527-09ac8c129b93 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:29.252626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6f7e3-0936-4a6d-a527-09ac8c129b93 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:29.253858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6f7e3-0936-4a6d-a527-09ac8c129b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:29.286314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3df6f7e3-0936-4a6d-a527-09ac8c129b93 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:29.288765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3df6f7e3-0936-4a6d-a527-09ac8c129b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:29.378492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:29.388013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:29.388914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:29.390355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] PATH_INFO: `/v3/users/2d32d61be57c40e488840c9b8a53de7d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:29.393804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d32d61be57c40e488840c9b8a53de7d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:29.400293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:29.490058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:29.562445 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 902/1789] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:29.563958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6f7e3-0936-4a6d-a527-09ac8c129b93 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:29.566285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3df6f7e3-0936-4a6d-a527-09ac8c129b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:29.593743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d32d61be57c40e488840c9b8a53de7d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.603173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:29.636197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6f7e3-0936-4a6d-a527-09ac8c129b93 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:29.639898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d32d61be57c40e488840c9b8a53de7d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.641664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d32d61be57c40e488840c9b8a53de7d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.646393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d32d61be57c40e488840c9b8a53de7d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.660346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:29.661028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:29.662969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:29.663498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] PATH_INFO: `/v3/projects/33594f3cce6d402ea7aaa031691203b4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:29.667290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=33594f3cce6d402ea7aaa031691203b4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:29.669956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-44157e2a-230e-4a34-8f6e-6de837aa52a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d32d61be57c40e488840c9b8a53de7d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.676944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:29.703599 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 888/1790] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:28 2022] DELETE /identity/v3/users/2d32d61be57c40e488840c9b8a53de7d => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:29.707415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea047a5-84f4-498d-8b7d-946b460a1367 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:29.727346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea047a5-84f4-498d-8b7d-946b460a1367 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:29.727763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea047a5-84f4-498d-8b7d-946b460a1367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:29.783856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fea047a5-84f4-498d-8b7d-946b460a1367 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:29.785525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fea047a5-84f4-498d-8b7d-946b460a1367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:29.854195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33594f3cce6d402ea7aaa031691203b4', 'request_id': 'req-0ae4904a-97e9-4e72-88fb-3e44d837b25d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.875165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33594f3cce6d402ea7aaa031691203b4', 'request_id': 'req-0ae4904a-97e9-4e72-88fb-3e44d837b25d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.891879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33594f3cce6d402ea7aaa031691203b4', 'request_id': 'req-0ae4904a-97e9-4e72-88fb-3e44d837b25d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.921409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] The token cache is being invalidate because project 33594f3cce6d402ea7aaa031691203b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:29.923816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.924718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0ae4904a-97e9-4e72-88fb-3e44d837b25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:29.934872 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 903/1791] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:29 2022] DELETE /identity/v3/projects/33594f3cce6d402ea7aaa031691203b4 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:29.936864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146a479-f221-489f-87cd-946fb479464d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:29.939558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146a479-f221-489f-87cd-946fb479464d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:29.940057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146a479-f221-489f-87cd-946fb479464d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:29.944684 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 904/1792] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:36:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:36:29.946538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3e8c9-2389-4441-918e-94b8f948a4c5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:29.948162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3e8c9-2389-4441-918e-94b8f948a4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:29.948621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3e8c9-2389-4441-918e-94b8f948a4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:30.008262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f9d3e8c9-2389-4441-918e-94b8f948a4c5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:30.009513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f9d3e8c9-2389-4441-918e-94b8f948a4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:30.083024 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 889/1793] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:30.083242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea047a5-84f4-498d-8b7d-946b460a1367 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:30.088011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fea047a5-84f4-498d-8b7d-946b460a1367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:30.113317 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 905/1794] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:30.114720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3e8c9-2389-4441-918e-94b8f948a4c5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:30.119717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea047a5-84f4-498d-8b7d-946b460a1367 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:30.128300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3e8c9-2389-4441-918e-94b8f948a4c5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:30.134172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:30.134741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:30.135151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:30.135463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:30.136123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:30.136442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:30.138122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:30.138614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:30.138915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:30.139241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:30.139529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] PATH_INFO: `/v3/users/aa754aeb51b54bbb9c5fc636a7fcb72a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:30.140901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa754aeb51b54bbb9c5fc636a7fcb72a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:30.143282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:30.159232 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 906/1795] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:30.160334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:30.171065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:30.253322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa754aeb51b54bbb9c5fc636a7fcb72a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:30.275225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa754aeb51b54bbb9c5fc636a7fcb72a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:30.276000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa754aeb51b54bbb9c5fc636a7fcb72a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:30.276645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa754aeb51b54bbb9c5fc636a7fcb72a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:30.304093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa754aeb51b54bbb9c5fc636a7fcb72a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:30.332839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd03889-2fc7-4f8d-b03b-6f0198173d7c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:30.345854 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 890/1796] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:30 2022] DELETE /identity/v3/users/aa754aeb51b54bbb9c5fc636a7fcb72a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:30.348735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:30.349486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc670550-ec1e-4dd6-a324-407fecc615f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:30.351819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc670550-ec1e-4dd6-a324-407fecc615f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:30.352276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc670550-ec1e-4dd6-a324-407fecc615f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:30.352710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc670550-ec1e-4dd6-a324-407fecc615f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:30.355549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:30.362548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fc670550-ec1e-4dd6-a324-407fecc615f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:30.386139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:30.411644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc670550-ec1e-4dd6-a324-407fecc615f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:30.416701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc670550-ec1e-4dd6-a324-407fecc615f9 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:30.444439 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 907/1797] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:30.446304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698311f0-4222-45a5-aa0a-fe5964f6e9c1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:30.446755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698311f0-4222-45a5-aa0a-fe5964f6e9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:30.449262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698311f0-4222-45a5-aa0a-fe5964f6e9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:30.499071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-698311f0-4222-45a5-aa0a-fe5964f6e9c1 None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:30.500615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-698311f0-4222-45a5-aa0a-fe5964f6e9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:30.507872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5341d-db9d-4e4b-9248-d811874f17d0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:30.526614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb23ca-c7a6-46eb-bea1-4186b527cef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:30.527246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb23ca-c7a6-46eb-bea1-4186b527cef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:30.527528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb23ca-c7a6-46eb-bea1-4186b527cef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:30.527800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb23ca-c7a6-46eb-bea1-4186b527cef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:30.528440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb23ca-c7a6-46eb-bea1-4186b527cef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:30.530801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb23ca-c7a6-46eb-bea1-4186b527cef9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:30.565978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:30.610561 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 891/1798] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:30 2022] POST /identity/v3/projects => generated 393 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:30.614320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb23ca-c7a6-46eb-bea1-4186b527cef9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:30.745467 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 908/1799] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:36:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:30.804505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb23ca-c7a6-46eb-bea1-4186b527cef9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:30.822142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29220f-7747-481a-8be0-c2f1d48824e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:30.823067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29220f-7747-481a-8be0-c2f1d48824e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:30.823067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29220f-7747-481a-8be0-c2f1d48824e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:30.823274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29220f-7747-481a-8be0-c2f1d48824e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:30.823921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29220f-7747-481a-8be0-c2f1d48824e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:30.825902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29220f-7747-481a-8be0-c2f1d48824e0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:30.898234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698311f0-4222-45a5-aa0a-fe5964f6e9c1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:30.900895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-698311f0-4222-45a5-aa0a-fe5964f6e9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:30.936131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698311f0-4222-45a5-aa0a-fe5964f6e9c1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:30.942598 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 892/1800] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:30 2022] POST /identity/v3/users => generated 297 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:30.954226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29220f-7747-481a-8be0-c2f1d48824e0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:30.994827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29220f-7747-481a-8be0-c2f1d48824e0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:31.012453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d523269-d91f-4c3f-944e-ed68fa685159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:31.013952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d523269-d91f-4c3f-944e-ed68fa685159 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:31.020165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d523269-d91f-4c3f-944e-ed68fa685159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:31.022660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d523269-d91f-4c3f-944e-ed68fa685159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:31.028539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d523269-d91f-4c3f-944e-ed68fa685159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:31.033170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d523269-d91f-4c3f-944e-ed68fa685159 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:31.109052 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 893/1801] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:31.111980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d523269-d91f-4c3f-944e-ed68fa685159 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:31.133227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c8d6-e8b2-4788-b68c-dfc69e851624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:31.138651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c8d6-e8b2-4788-b68c-dfc69e851624 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:31.140474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c8d6-e8b2-4788-b68c-dfc69e851624 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:31.141487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c8d6-e8b2-4788-b68c-dfc69e851624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:31.168483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d523269-d91f-4c3f-944e-ed68fa685159 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:31.186713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8c8d6-e8b2-4788-b68c-dfc69e851624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:31.196055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8c8d6-e8b2-4788-b68c-dfc69e851624 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:31.199063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee8b1f-e284-4870-8a1a-93499af85736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:31.199701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee8b1f-e284-4870-8a1a-93499af85736 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:31.199983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee8b1f-e284-4870-8a1a-93499af85736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:31.200256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee8b1f-e284-4870-8a1a-93499af85736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:31.200918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee8b1f-e284-4870-8a1a-93499af85736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:31.202570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee8b1f-e284-4870-8a1a-93499af85736 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:31.253289 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 909/1802] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:31.257575 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 894/1803] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:31.265895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c8d6-e8b2-4788-b68c-dfc69e851624 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:31.331791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c8d6-e8b2-4788-b68c-dfc69e851624 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:31.382051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:31.384933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:31.386837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:31.397655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] PATH_INFO: `/v3/projects/253ce34785e8458ba38c1036b7687c63/users/0763c8c77a8644d1ba30aa62b4b56c9b/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:31.492827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=253ce34785e8458ba38c1036b7687c63, user_id=0763c8c77a8644d1ba30aa62b4b56c9b, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:31.496220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:31.551797 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 910/1804] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:31 2022] PUT /identity/v3/projects/253ce34785e8458ba38c1036b7687c63/users/0763c8c77a8644d1ba30aa62b4b56c9b/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:31.558353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14c061-d746-4675-9372-a5a136aad565 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:31.558873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14c061-d746-4675-9372-a5a136aad565 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:31.559561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14c061-d746-4675-9372-a5a136aad565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:31.693415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:31.697859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:31.704408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-eb14c061-d746-4675-9372-a5a136aad565 None None] MFA Rules not processed for user `0763c8c77a8644d1ba30aa62b4b56c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:31.708234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eb14c061-d746-4675-9372-a5a136aad565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:31.948219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c3fde-2913-40f2-bcef-7d5f16b564ae admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:31.980684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:31.982082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:31.983218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:31.983709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:31.984200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] PATH_INFO: `/v3/projects/0d563afa0afd4dae894d8256e838aeb5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:31.988047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d563afa0afd4dae894d8256e838aeb5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:32.004144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:32.040537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:32.073198 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 895/1805] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:32.094095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14c061-d746-4675-9372-a5a136aad565 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:32.136487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d563afa0afd4dae894d8256e838aeb5', 'request_id': 'req-a1682bb9-8645-47b2-9acc-b586f4cace50'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:32.143305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14c061-d746-4675-9372-a5a136aad565 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:32.156272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d563afa0afd4dae894d8256e838aeb5', 'request_id': 'req-a1682bb9-8645-47b2-9acc-b586f4cace50'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:32.169115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d563afa0afd4dae894d8256e838aeb5', 'request_id': 'req-a1682bb9-8645-47b2-9acc-b586f4cace50'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:32.205014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] The token cache is being invalidate because project 0d563afa0afd4dae894d8256e838aeb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:32.205326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:32.206905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a1682bb9-8645-47b2-9acc-b586f4cace50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:32.227324 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 911/1806] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:31 2022] DELETE /identity/v3/projects/0d563afa0afd4dae894d8256e838aeb5 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:32.230019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26298306-ccc8-4295-9268-fec267b6c8aa None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:32.230323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26298306-ccc8-4295-9268-fec267b6c8aa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:32.230604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26298306-ccc8-4295-9268-fec267b6c8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:32.282930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-26298306-ccc8-4295-9268-fec267b6c8aa None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:32.285072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-26298306-ccc8-4295-9268-fec267b6c8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:32.353078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:32.357321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:32.357606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:32.358209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:32.359157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:32.363990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:32.463602 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 896/1807] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:32 2022] POST /identity/v3/projects => generated 393 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:32.465226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:32.468017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:32.475843 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 912/1808] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:32.478908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcddf6-1186-4058-b316-7f9dda3194a8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:32.487834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcddf6-1186-4058-b316-7f9dda3194a8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:32.487834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcddf6-1186-4058-b316-7f9dda3194a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:32.572500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f7fcddf6-1186-4058-b316-7f9dda3194a8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:32.578128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7fcddf6-1186-4058-b316-7f9dda3194a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:32.768644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5f56a-940a-4a50-a76d-303ac4e5cb61 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:32.786411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00623e0-8b9d-41d4-831b-7795c771e174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:32.790518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00623e0-8b9d-41d4-831b-7795c771e174 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:32.791114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00623e0-8b9d-41d4-831b-7795c771e174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:32.791744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00623e0-8b9d-41d4-831b-7795c771e174 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:32.793432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00623e0-8b9d-41d4-831b-7795c771e174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:32.824599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00623e0-8b9d-41d4-831b-7795c771e174 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:32.845984 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 913/1809] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:32.847422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcddf6-1186-4058-b316-7f9dda3194a8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:32.850392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7fcddf6-1186-4058-b316-7f9dda3194a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:32.906319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcddf6-1186-4058-b316-7f9dda3194a8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:32.924777 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 897/1810] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:32 2022] POST /identity/v3/users => generated 303 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:32.924777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00623e0-8b9d-41d4-831b-7795c771e174 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:32.936204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:32.937840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:32.940296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:32.941059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] PATH_INFO: `/v3/projects/4c66a4ab9bf9477ca5236dedb67c4617` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:32.945093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c66a4ab9bf9477ca5236dedb67c4617)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:32.958897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:32.964234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00623e0-8b9d-41d4-831b-7795c771e174 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:32.993353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:32.994042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:32.994329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:32.994584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:32.995253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:33.000427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:33.004497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:33.057029 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 898/1811] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:33.057924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:33.073152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:33.090480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7b652-e837-4629-9048-0ece24bf0c73 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:33.098542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c66a4ab9bf9477ca5236dedb67c4617', 'request_id': 'req-bb1930ff-c896-48ae-9daf-10c357edaabe'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.115941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c66a4ab9bf9477ca5236dedb67c4617', 'request_id': 'req-bb1930ff-c896-48ae-9daf-10c357edaabe'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.139579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c66a4ab9bf9477ca5236dedb67c4617', 'request_id': 'req-bb1930ff-c896-48ae-9daf-10c357edaabe'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.165335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] The token cache is being invalidate because project 4c66a4ab9bf9477ca5236dedb67c4617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:33.166316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.168715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.171127 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 914/1812] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:32 2022] DELETE /identity/v3/projects/4c66a4ab9bf9477ca5236dedb67c4617 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:33.173975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:33.177267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:33.200996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:33.233009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:33.242945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:33.243410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:33.243947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:33.244843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:33.246710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:33.264583 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 899/1813] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:33.268954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:33.272037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:33.316495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d89ba-0662-46fa-8c95-8b8b6896cb2d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:33.323374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1930ff-c896-48ae-9daf-10c357edaabe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:33.337132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:33.338286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:33.338803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:33.339126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:33.341312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:33.343981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:33.348571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:33.350644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:33.351179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:33.351552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] PATH_INFO: `/v3/users/6c023db0bd084ae2829ec7aa11a6dc4d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:33.354720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c023db0bd084ae2829ec7aa11a6dc4d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:33.359258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:33.441071 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 915/1814] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:33 2022] POST /identity/v3/projects => generated 375 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:33.443799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:33.452196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:33.496479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c023db0bd084ae2829ec7aa11a6dc4d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.514457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c023db0bd084ae2829ec7aa11a6dc4d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.515283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c023db0bd084ae2829ec7aa11a6dc4d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.515934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c023db0bd084ae2829ec7aa11a6dc4d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.531469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c023db0bd084ae2829ec7aa11a6dc4d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:33.569510 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 900/1815] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:33 2022] DELETE /identity/v3/users/6c023db0bd084ae2829ec7aa11a6dc4d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:33.571969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:33.577826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:33.623169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2016500-2033-45e8-b72e-a221a2a79a2c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:33.752720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:33.766600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6539e95-dd9b-44ed-be7e-d2e35490ea04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:33.767842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539e95-dd9b-44ed-be7e-d2e35490ea04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:33.768459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539e95-dd9b-44ed-be7e-d2e35490ea04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:33.769901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6539e95-dd9b-44ed-be7e-d2e35490ea04 admin admin] PATH_INFO: `/v3/projects/b6f449bc790d4b7b95d314de67e10a75/users/4ae41689ca15468e8813819a2531ff72/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:33.781926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7005ef-633a-4964-8602-65230405b0c9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:33.789539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:33.790252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:33.791421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:33.791720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:33.793801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:33.798358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:33.806269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6539e95-dd9b-44ed-be7e-d2e35490ea04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f449bc790d4b7b95d314de67e10a75, user_id=4ae41689ca15468e8813819a2531ff72, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:33.810785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6539e95-dd9b-44ed-be7e-d2e35490ea04 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:33.840067 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 916/1816] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:33 2022] PUT /identity/v3/projects/b6f449bc790d4b7b95d314de67e10a75/users/4ae41689ca15468e8813819a2531ff72/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:33.841259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c246c62-0d63-4aba-9d19-5d9eda543456 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:33.841549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c246c62-0d63-4aba-9d19-5d9eda543456 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:33.841849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c246c62-0d63-4aba-9d19-5d9eda543456 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:33.848505 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 901/1817] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:33 2022] POST /identity/v3/users => generated 288 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:33.850173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:33.850863 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 917/1818] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:36:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:36:33.852460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:33.854794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c246c62-0d63-4aba-9d19-5d9eda543456 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:33.877177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b4704-3f3c-4357-b53a-773104191393 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:33.977783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:33.991740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c246c62-0d63-4aba-9d19-5d9eda543456 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:34.002336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e24e2-0dc0-4938-8671-9c9bc9a8e8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:34.007367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e24e2-0dc0-4938-8671-9c9bc9a8e8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:34.007918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e24e2-0dc0-4938-8671-9c9bc9a8e8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:34.008797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fc965-5e02-4e23-96ed-93f5d7cad7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:34.009244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e24e2-0dc0-4938-8671-9c9bc9a8e8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:34.010069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e24e2-0dc0-4938-8671-9c9bc9a8e8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:34.010734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc965-5e02-4e23-96ed-93f5d7cad7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:34.011449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc965-5e02-4e23-96ed-93f5d7cad7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:34.012488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc965-5e02-4e23-96ed-93f5d7cad7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:34.013222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fc965-5e02-4e23-96ed-93f5d7cad7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:34.014099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e24e2-0dc0-4938-8671-9c9bc9a8e8c4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:34.015879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fc965-5e02-4e23-96ed-93f5d7cad7ce admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:34.045609 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 918/1819] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:34.046468 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 902/1820] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:34.048000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bef5dd-91a5-4144-b84b-c6684bacd45d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:34.048639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bef5dd-91a5-4144-b84b-c6684bacd45d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:34.048989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bef5dd-91a5-4144-b84b-c6684bacd45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:34.054585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e24e2-0dc0-4938-8671-9c9bc9a8e8c4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:34.072142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e24e2-0dc0-4938-8671-9c9bc9a8e8c4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:34.088364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bba09-361d-4a98-a96d-e3c75da5b058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:34.090103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bba09-361d-4a98-a96d-e3c75da5b058 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:34.091044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bba09-361d-4a98-a96d-e3c75da5b058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:34.092491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bba09-361d-4a98-a96d-e3c75da5b058 admin admin] PATH_INFO: `/v3/projects/b6f449bc790d4b7b95d314de67e10a75/users/4ae41689ca15468e8813819a2531ff72/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:34.098301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13bba09-361d-4a98-a96d-e3c75da5b058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f449bc790d4b7b95d314de67e10a75, user_id=4ae41689ca15468e8813819a2531ff72, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:34.101181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13bba09-361d-4a98-a96d-e3c75da5b058 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:34.102946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f9bef5dd-91a5-4144-b84b-c6684bacd45d None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:34.104206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f9bef5dd-91a5-4144-b84b-c6684bacd45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:34.149082 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 903/1821] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:34 2022] PUT /identity/v3/projects/b6f449bc790d4b7b95d314de67e10a75/users/4ae41689ca15468e8813819a2531ff72/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:34.150172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bba09-361d-4a98-a96d-e3c75da5b058 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:34.295048 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 919/1822] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:36:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:34.297229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c2f0a-4894-491d-b396-7b9b6650569f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:34.297522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c2f0a-4894-491d-b396-7b9b6650569f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:34.301416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c2f0a-4894-491d-b396-7b9b6650569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:34.316235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13bba09-361d-4a98-a96d-e3c75da5b058 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:34.345180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a9fd0-b839-4e07-b294-78928a9cc7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:34.350830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a9fd0-b839-4e07-b294-78928a9cc7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:34.351217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a9fd0-b839-4e07-b294-78928a9cc7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:34.351502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a9fd0-b839-4e07-b294-78928a9cc7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:34.352267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a9fd0-b839-4e07-b294-78928a9cc7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:34.354046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a9fd0-b839-4e07-b294-78928a9cc7b9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:34.437289 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 904/1823] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:34.441532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a9fd0-b839-4e07-b294-78928a9cc7b9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:34.454142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dc2c2f0a-4894-491d-b396-7b9b6650569f None None] MFA Rules not processed for user `4ae41689ca15468e8813819a2531ff72`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:34.455524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dc2c2f0a-4894-491d-b396-7b9b6650569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:34.486646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a9fd0-b839-4e07-b294-78928a9cc7b9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:34.506464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9a6b-aaa4-4cbd-891f-c3ba3ef9eab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:34.507162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9a6b-aaa4-4cbd-891f-c3ba3ef9eab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:34.507433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9a6b-aaa4-4cbd-891f-c3ba3ef9eab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:34.507712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9a6b-aaa4-4cbd-891f-c3ba3ef9eab6 admin admin] PATH_INFO: `/v3/projects/d2e34f0da33544549e5636cd9f815a90/users/d946f45a27054a0c8c7be4ea3d0d9cfb/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:34.575561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde9a6b-aaa4-4cbd-891f-c3ba3ef9eab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e34f0da33544549e5636cd9f815a90, user_id=d946f45a27054a0c8c7be4ea3d0d9cfb, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:34.578457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde9a6b-aaa4-4cbd-891f-c3ba3ef9eab6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:34.605030 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 905/1824] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:34 2022] PUT /identity/v3/projects/d2e34f0da33544549e5636cd9f815a90/users/d946f45a27054a0c8c7be4ea3d0d9cfb/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:34.620406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321d77c-e573-4c31-af78-e76fa3f3f078 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:34.621094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321d77c-e573-4c31-af78-e76fa3f3f078 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:34.624110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321d77c-e573-4c31-af78-e76fa3f3f078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:34.735965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:34.748521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b321d77c-e573-4c31-af78-e76fa3f3f078 None None] MFA Rules not processed for user `d946f45a27054a0c8c7be4ea3d0d9cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:34.750620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b321d77c-e573-4c31-af78-e76fa3f3f078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:34.773797 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 920/1825] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:34 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:34.783402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c2f0a-4894-491d-b396-7b9b6650569f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:34.786807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dc2c2f0a-4894-491d-b396-7b9b6650569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:34.968616 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 906/1826] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:34.969822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321d77c-e573-4c31-af78-e76fa3f3f078 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:34.972114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b321d77c-e573-4c31-af78-e76fa3f3f078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:35.027019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c2f0a-4894-491d-b396-7b9b6650569f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:35.080615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:35.160557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321d77c-e573-4c31-af78-e76fa3f3f078 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:35.164543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:35.165389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:35.165483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:35.165755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:35.228729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2683d-50fd-4132-a289-d6de3a189010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:35.230208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2683d-50fd-4132-a289-d6de3a189010 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:35.231095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2683d-50fd-4132-a289-d6de3a189010 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:35.231656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2683d-50fd-4132-a289-d6de3a189010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:35.244596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2683d-50fd-4132-a289-d6de3a189010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:35.247600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2683d-50fd-4132-a289-d6de3a189010 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:35.265867 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 921/1827] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:35.267366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2683d-50fd-4132-a289-d6de3a189010 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:35.386442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:35.387971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:35.398246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2683d-50fd-4132-a289-d6de3a189010 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:35.405076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6478d5-b653-4423-b9e3-708da7422c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:35.405315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6478d5-b653-4423-b9e3-708da7422c85 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:35.405509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6478d5-b653-4423-b9e3-708da7422c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:35.405841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6478d5-b653-4423-b9e3-708da7422c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:35.407045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6478d5-b653-4423-b9e3-708da7422c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:35.411167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6478d5-b653-4423-b9e3-708da7422c85 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:35.468719 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 907/1828] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:35.475274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:35.485155 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 922/1829] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:35 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:35.485292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:35.487234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6478d5-b653-4423-b9e3-708da7422c85 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:35.501136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:35.538932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a8875-ced5-4d7a-9a5f-590a95ad8530 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:35.703039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:35.703796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:35.704144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:35.704483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] PATH_INFO: `/v3/projects/854c589c26594f7090a7e2ab73b8744f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:35.708260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=854c589c26594f7090a7e2ab73b8744f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:35.719614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:35.747813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6478d5-b653-4423-b9e3-708da7422c85 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:35.768121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06533e3-9ef7-484c-9867-413c64368783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:35.769897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06533e3-9ef7-484c-9867-413c64368783 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:35.770552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06533e3-9ef7-484c-9867-413c64368783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:35.770768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06533e3-9ef7-484c-9867-413c64368783 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:35.772657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06533e3-9ef7-484c-9867-413c64368783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:35.775261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06533e3-9ef7-484c-9867-413c64368783 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:35.842959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854c589c26594f7090a7e2ab73b8744f', 'request_id': 'req-81547c9f-2a2e-4398-a723-7bbc5c84ab68'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:35.846709 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 923/1830] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:35 2022] POST /identity/v3/users => generated 295 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:35.848282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06533e3-9ef7-484c-9867-413c64368783 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:35.860996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854c589c26594f7090a7e2ab73b8744f', 'request_id': 'req-81547c9f-2a2e-4398-a723-7bbc5c84ab68'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:35.876032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854c589c26594f7090a7e2ab73b8744f', 'request_id': 'req-81547c9f-2a2e-4398-a723-7bbc5c84ab68'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:35.909882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] The token cache is being invalidate because project 854c589c26594f7090a7e2ab73b8744f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:35.911290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:35.912527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:35.915603 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 908/1831] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:35 2022] DELETE /identity/v3/projects/854c589c26594f7090a7e2ab73b8744f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:35.916465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:35.918576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:35.939258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:36.067471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06533e3-9ef7-484c-9867-413c64368783 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:36.101157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547c9f-2a2e-4398-a723-7bbc5c84ab68 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:36.114585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a03bee-8f94-4daa-a5c5-63e86bb7010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:36.116657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a03bee-8f94-4daa-a5c5-63e86bb7010f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.117090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a03bee-8f94-4daa-a5c5-63e86bb7010f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.117385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a03bee-8f94-4daa-a5c5-63e86bb7010f admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.118103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a03bee-8f94-4daa-a5c5-63e86bb7010f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:36.124733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a03bee-8f94-4daa-a5c5-63e86bb7010f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:36.153641 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 909/1832] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:36.170766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186df45-bda8-42ba-8ae0-46b61fa409db None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.172360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186df45-bda8-42ba-8ae0-46b61fa409db None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.172774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186df45-bda8-42ba-8ae0-46b61fa409db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.182560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93786362-701b-42bd-a7db-3ddf62349ae3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:36.184029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93786362-701b-42bd-a7db-3ddf62349ae3 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.185059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93786362-701b-42bd-a7db-3ddf62349ae3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.185394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93786362-701b-42bd-a7db-3ddf62349ae3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.188970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-93786362-701b-42bd-a7db-3ddf62349ae3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:36.275152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b186df45-bda8-42ba-8ae0-46b61fa409db None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:36.281084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b186df45-bda8-42ba-8ae0-46b61fa409db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:36.350436 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 910/1833] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:36.350844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186df45-bda8-42ba-8ae0-46b61fa409db None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:36.368229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93786362-701b-42bd-a7db-3ddf62349ae3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:36.380492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93786362-701b-42bd-a7db-3ddf62349ae3 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:36.400666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186df45-bda8-42ba-8ae0-46b61fa409db None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:36.416950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee135c6-d69f-405c-ad83-7701f3fffe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:36.417220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee135c6-d69f-405c-ad83-7701f3fffe83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.417533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee135c6-d69f-405c-ad83-7701f3fffe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.417848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee135c6-d69f-405c-ad83-7701f3fffe83 admin admin] PATH_INFO: `/v3/projects/9cf0853d9cd24f07b59d94e772675fa9/users/8607a1406a26485bb208a0fc2af331b1/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.439584 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 924/1834] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:35 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:36.441183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52735a-20ae-4b20-9701-ce71fe8992b9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.441396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52735a-20ae-4b20-9701-ce71fe8992b9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.447982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52735a-20ae-4b20-9701-ce71fe8992b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.475193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:36.484229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee135c6-d69f-405c-ad83-7701f3fffe83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf0853d9cd24f07b59d94e772675fa9, user_id=8607a1406a26485bb208a0fc2af331b1, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:36.493373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee135c6-d69f-405c-ad83-7701f3fffe83 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:36.515933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ca52735a-20ae-4b20-9701-ce71fe8992b9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:36.525077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ca52735a-20ae-4b20-9701-ce71fe8992b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:36.534688 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 911/1835] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:36 2022] PUT /identity/v3/projects/9cf0853d9cd24f07b59d94e772675fa9/users/8607a1406a26485bb208a0fc2af331b1/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:36.539725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee135c6-d69f-405c-ad83-7701f3fffe83 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:36.692404 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 925/1836] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:36.697789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52735a-20ae-4b20-9701-ce71fe8992b9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:36.702692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee135c6-d69f-405c-ad83-7701f3fffe83 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:36.710806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8e979-4fa7-48fe-963c-0bb490192c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:36.711267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52735a-20ae-4b20-9701-ce71fe8992b9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:36.711558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8e979-4fa7-48fe-963c-0bb490192c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.711845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8e979-4fa7-48fe-963c-0bb490192c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.712122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8e979-4fa7-48fe-963c-0bb490192c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.712791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8e979-4fa7-48fe-963c-0bb490192c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:36.714427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8e979-4fa7-48fe-963c-0bb490192c01 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:36.728761 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 912/1837] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:36.733959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8e979-4fa7-48fe-963c-0bb490192c01 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:36.734630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:36.738041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.738329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.741233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.743100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:36.750390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:36.750390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:36.762347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8e979-4fa7-48fe-963c-0bb490192c01 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:36.779512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f5ba6-c0dd-44c9-a9cc-b5ae675e609e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:36.780728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5ba6-c0dd-44c9-a9cc-b5ae675e609e admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.781315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5ba6-c0dd-44c9-a9cc-b5ae675e609e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.781729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f5ba6-c0dd-44c9-a9cc-b5ae675e609e admin admin] PATH_INFO: `/v3/projects/9cf0853d9cd24f07b59d94e772675fa9/users/8607a1406a26485bb208a0fc2af331b1/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.782119 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 926/1838] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:36.786735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:36.792455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5ba6-c0dd-44c9-a9cc-b5ae675e609e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf0853d9cd24f07b59d94e772675fa9, user_id=8607a1406a26485bb208a0fc2af331b1, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:36.802542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f5ba6-c0dd-44c9-a9cc-b5ae675e609e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:36.805046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e684d-855a-40c4-9eae-bb6f7bea5944 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:36.810643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad70ea8-897e-4fa3-b032-89d8b84c901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:36.811497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad70ea8-897e-4fa3-b032-89d8b84c901b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.811873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad70ea8-897e-4fa3-b032-89d8b84c901b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.812245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad70ea8-897e-4fa3-b032-89d8b84c901b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.813407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad70ea8-897e-4fa3-b032-89d8b84c901b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:36.816112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad70ea8-897e-4fa3-b032-89d8b84c901b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:36.837073 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 913/1839] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:36 2022] PUT /identity/v3/projects/9cf0853d9cd24f07b59d94e772675fa9/users/8607a1406a26485bb208a0fc2af331b1/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:36.848985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae6cea-a753-4252-9839-54fc3db81cc1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:36.849371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae6cea-a753-4252-9839-54fc3db81cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:36.850396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae6cea-a753-4252-9839-54fc3db81cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:36.861677 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 927/1840] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:36 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:36.864183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad70ea8-897e-4fa3-b032-89d8b84c901b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:36.941808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-43ae6cea-a753-4252-9839-54fc3db81cc1 None None] MFA Rules not processed for user `8607a1406a26485bb208a0fc2af331b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:36.943624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-43ae6cea-a753-4252-9839-54fc3db81cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:36.992500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad70ea8-897e-4fa3-b032-89d8b84c901b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:36.999496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc00ef-d449-4a9e-9826-7c2af5d4b56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.001956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc00ef-d449-4a9e-9826-7c2af5d4b56c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.002346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc00ef-d449-4a9e-9826-7c2af5d4b56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.002735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc00ef-d449-4a9e-9826-7c2af5d4b56c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.003476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc00ef-d449-4a9e-9826-7c2af5d4b56c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.007177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc00ef-d449-4a9e-9826-7c2af5d4b56c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.047430 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 928/1841] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:36 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:37.053404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc00ef-d449-4a9e-9826-7c2af5d4b56c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.055302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:37.077360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc00ef-d449-4a9e-9826-7c2af5d4b56c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.092424 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 914/1842] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:37.094928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e09795-3355-4fbd-914c-14419d04daca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.095602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09795-3355-4fbd-914c-14419d04daca admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.095955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09795-3355-4fbd-914c-14419d04daca admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.104244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09795-3355-4fbd-914c-14419d04daca admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.105775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae6cea-a753-4252-9839-54fc3db81cc1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.107175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e09795-3355-4fbd-914c-14419d04daca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.108165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-43ae6cea-a753-4252-9839-54fc3db81cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:37.112022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e09795-3355-4fbd-914c-14419d04daca admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.141150 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 929/1843] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:37.143754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e09795-3355-4fbd-914c-14419d04daca admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.175497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e09795-3355-4fbd-914c-14419d04daca admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.182558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286f694-d8f5-4807-a870-7867855cecb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.183083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f694-d8f5-4807-a870-7867855cecb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.183423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f694-d8f5-4807-a870-7867855cecb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.183712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f694-d8f5-4807-a870-7867855cecb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.184555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286f694-d8f5-4807-a870-7867855cecb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.186487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286f694-d8f5-4807-a870-7867855cecb8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.200042 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 930/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:37.203989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286f694-d8f5-4807-a870-7867855cecb8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.211007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae6cea-a753-4252-9839-54fc3db81cc1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.217380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286f694-d8f5-4807-a870-7867855cecb8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.219696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.220531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.221037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.221575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.223613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7513a-e986-4579-be54-f4792ce7b187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.224269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:37.224533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7513a-e986-4579-be54-f4792ce7b187 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.225002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7513a-e986-4579-be54-f4792ce7b187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.225341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7513a-e986-4579-be54-f4792ce7b187 admin admin] PATH_INFO: `/v3/projects/bb75584943b64005af766980d3d20af0/users/f585d2dbfdef413c9eab2459ab9b89bb/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.242058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7513a-e986-4579-be54-f4792ce7b187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb75584943b64005af766980d3d20af0, user_id=f585d2dbfdef413c9eab2459ab9b89bb, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.245110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7513a-e986-4579-be54-f4792ce7b187 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.263389 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 931/1845] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:37 2022] PUT /identity/v3/projects/bb75584943b64005af766980d3d20af0/users/f585d2dbfdef413c9eab2459ab9b89bb/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:37.275391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703164cd-36de-4f0b-9497-17dbf8c1bfb3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.275603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703164cd-36de-4f0b-9497-17dbf8c1bfb3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.275904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703164cd-36de-4f0b-9497-17dbf8c1bfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.297329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:37.299138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.300614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.306590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-703164cd-36de-4f0b-9497-17dbf8c1bfb3 None None] MFA Rules not processed for user `f585d2dbfdef413c9eab2459ab9b89bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:37.308161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-703164cd-36de-4f0b-9497-17dbf8c1bfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:37.310404 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 915/1846] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:37.373984 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 932/1847] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:37.378550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.389582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15764fd4-57d4-4220-92c2-250db561fc29 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.465893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3705f87-9749-4caf-8116-9e4b143683a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.466515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3705f87-9749-4caf-8116-9e4b143683a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.466799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3705f87-9749-4caf-8116-9e4b143683a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.467083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3705f87-9749-4caf-8116-9e4b143683a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.467719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3705f87-9749-4caf-8116-9e4b143683a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.469647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3705f87-9749-4caf-8116-9e4b143683a9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.499119 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 916/1848] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:37 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:37.503307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703164cd-36de-4f0b-9497-17dbf8c1bfb3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.514504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703164cd-36de-4f0b-9497-17dbf8c1bfb3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.598498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b837-7952-482f-b5e9-f1862dbdad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.599899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b837-7952-482f-b5e9-f1862dbdad95 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.601126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b837-7952-482f-b5e9-f1862dbdad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.601653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893b837-7952-482f-b5e9-f1862dbdad95 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.602689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893b837-7952-482f-b5e9-f1862dbdad95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.605922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893b837-7952-482f-b5e9-f1862dbdad95 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.632565 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 933/1849] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:37 2022] POST /identity/v3/users => generated 300 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:37.636854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3705f87-9749-4caf-8116-9e4b143683a9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.647767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3705f87-9749-4caf-8116-9e4b143683a9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.652513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6a28b-9e6a-4cde-9cf7-ddfdb6ee50a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.653123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a28b-9e6a-4cde-9cf7-ddfdb6ee50a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.653401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a28b-9e6a-4cde-9cf7-ddfdb6ee50a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.653697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a28b-9e6a-4cde-9cf7-ddfdb6ee50a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.654460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6a28b-9e6a-4cde-9cf7-ddfdb6ee50a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.656065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d6a28b-9e6a-4cde-9cf7-ddfdb6ee50a2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.670921 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 917/1850] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:37.675078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b837-7952-482f-b5e9-f1862dbdad95 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.685975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893b837-7952-482f-b5e9-f1862dbdad95 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.690984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50b2f3-0e2e-4308-bc20-7dca9ddb212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.691650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50b2f3-0e2e-4308-bc20-7dca9ddb212d admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.691911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50b2f3-0e2e-4308-bc20-7dca9ddb212d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.692186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50b2f3-0e2e-4308-bc20-7dca9ddb212d admin admin] PATH_INFO: `/v3/projects/48b6c97dcba94945ab5878ba4a663edf/users/613c89130e2349c6bbc3e44619735202/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.709507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50b2f3-0e2e-4308-bc20-7dca9ddb212d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b6c97dcba94945ab5878ba4a663edf, user_id=613c89130e2349c6bbc3e44619735202, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.712334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50b2f3-0e2e-4308-bc20-7dca9ddb212d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.730149 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 934/1851] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:37 2022] PUT /identity/v3/projects/48b6c97dcba94945ab5878ba4a663edf/users/613c89130e2349c6bbc3e44619735202/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:37.736405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6a28b-9e6a-4cde-9cf7-ddfdb6ee50a2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.745098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6a28b-9e6a-4cde-9cf7-ddfdb6ee50a2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.759518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:37.817415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50c1ae-f0d1-4a06-8088-966b86f3231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.818267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50c1ae-f0d1-4a06-8088-966b86f3231f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.819370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50c1ae-f0d1-4a06-8088-966b86f3231f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.819756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50c1ae-f0d1-4a06-8088-966b86f3231f admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.820526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50c1ae-f0d1-4a06-8088-966b86f3231f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.822327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50c1ae-f0d1-4a06-8088-966b86f3231f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.833315 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 918/1852] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:37.838236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50b2f3-0e2e-4308-bc20-7dca9ddb212d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:37.848496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50b2f3-0e2e-4308-bc20-7dca9ddb212d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:37.853844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:37.854500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.854781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.855052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] PATH_INFO: `/v3/projects/48b6c97dcba94945ab5878ba4a663edf/users/613c89130e2349c6bbc3e44619735202/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.856952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b6c97dcba94945ab5878ba4a663edf, user_id=613c89130e2349c6bbc3e44619735202, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:37.859791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:37.870570 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 935/1853] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:37 2022] PUT /identity/v3/projects/48b6c97dcba94945ab5878ba4a663edf/users/613c89130e2349c6bbc3e44619735202/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:37.882680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebcf9c-7bf0-4967-9940-1317b4323e4c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:37.883107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebcf9c-7bf0-4967-9940-1317b4323e4c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:37.883390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebcf9c-7bf0-4967-9940-1317b4323e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:37.914210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-dcebcf9c-7bf0-4967-9940-1317b4323e4c None None] MFA Rules not processed for user `613c89130e2349c6bbc3e44619735202`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:37.915440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dcebcf9c-7bf0-4967-9940-1317b4323e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:37.999593 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 919/1854] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:38.011610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:38.013354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:38.080318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd84f28-02a8-456c-97cc-a1d0e31812e8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:38.084409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e49b9-9c31-463a-ab1a-99ca46a3d03e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:38.085003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e49b9-9c31-463a-ab1a-99ca46a3d03e service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:38.085285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e49b9-9c31-463a-ab1a-99ca46a3d03e service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:38.085552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e49b9-9c31-463a-ab1a-99ca46a3d03e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:38.092895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:38.097764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e49b9-9c31-463a-ab1a-99ca46a3d03e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:38.099693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e49b9-9c31-463a-ab1a-99ca46a3d03e service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:38.110354 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 936/1855] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:38.282722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebcf9c-7bf0-4967-9940-1317b4323e4c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:38.382829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebcf9c-7bf0-4967-9940-1317b4323e4c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:38.387451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:38.388172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:38.388586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:38.388859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] PATH_INFO: `/v3/users/f585d2dbfdef413c9eab2459ab9b89bb` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:38.390150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f585d2dbfdef413c9eab2459ab9b89bb)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:38.392137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:38.453177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:38.461739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f585d2dbfdef413c9eab2459ab9b89bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:38.469752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f585d2dbfdef413c9eab2459ab9b89bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:38.470524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f585d2dbfdef413c9eab2459ab9b89bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:38.471127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f585d2dbfdef413c9eab2459ab9b89bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:38.481556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f585d2dbfdef413c9eab2459ab9b89bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:38.501655 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 920/1856] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:38 2022] DELETE /identity/v3/users/f585d2dbfdef413c9eab2459ab9b89bb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:38.506693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21054dc-9086-49c2-8aff-fd7b56eb9531 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:38.507072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21054dc-9086-49c2-8aff-fd7b56eb9531 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:38.507349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21054dc-9086-49c2-8aff-fd7b56eb9531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:38.526773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f21054dc-9086-49c2-8aff-fd7b56eb9531 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:38.527973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f21054dc-9086-49c2-8aff-fd7b56eb9531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:38.608606 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 937/1857] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:38.622390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:38.623836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:38.703795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7332-f5fb-4d35-bc68-cea1f893d214 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:38.707544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03793c41-1a54-4bfe-a723-f415049fdf5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:38.708157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03793c41-1a54-4bfe-a723-f415049fdf5c service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:38.708436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03793c41-1a54-4bfe-a723-f415049fdf5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:38.708715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03793c41-1a54-4bfe-a723-f415049fdf5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:38.719117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03793c41-1a54-4bfe-a723-f415049fdf5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:38.720553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03793c41-1a54-4bfe-a723-f415049fdf5c service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:38.730787 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 921/1858] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:39.035276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21054dc-9086-49c2-8aff-fd7b56eb9531 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:39.037916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f21054dc-9086-49c2-8aff-fd7b56eb9531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:39.053240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21054dc-9086-49c2-8aff-fd7b56eb9531 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:39.071971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:39.071971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:39.073191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:39.073533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] PATH_INFO: `/v3/projects/bb75584943b64005af766980d3d20af0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:39.075499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb75584943b64005af766980d3d20af0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:39.078627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:39.164870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb75584943b64005af766980d3d20af0', 'request_id': 'req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.174692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb75584943b64005af766980d3d20af0', 'request_id': 'req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.182685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb75584943b64005af766980d3d20af0', 'request_id': 'req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.193723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] The token cache is being invalidate because project bb75584943b64005af766980d3d20af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:39.194005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.194706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.197044 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 938/1859] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:39 2022] DELETE /identity/v3/projects/bb75584943b64005af766980d3d20af0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:39.200893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03793c41-1a54-4bfe-a723-f415049fdf5c service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:39.212452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03793c41-1a54-4bfe-a723-f415049fdf5c service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:39.305382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:39.306048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:39.306327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:39.306598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] PATH_INFO: `/v3/users/613c89130e2349c6bbc3e44619735202` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:39.307714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] RBAC: Authorizing `identity:delete_user(user_id=613c89130e2349c6bbc3e44619735202)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:39.309692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:39.359196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '613c89130e2349c6bbc3e44619735202'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.368764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '613c89130e2349c6bbc3e44619735202'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.370165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '613c89130e2349c6bbc3e44619735202'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.371555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '613c89130e2349c6bbc3e44619735202'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.377309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:39.389221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b4c0d413-9d99-4729-bee9-569782f4c30b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '613c89130e2349c6bbc3e44619735202'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:39.425358 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 922/1860] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:39 2022] DELETE /identity/v3/users/613c89130e2349c6bbc3e44619735202 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:39.854335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:39.857977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:39.905972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc96168-a93c-4f22-a2e1-f5f5e998fc4f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:40.006551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:40.008647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.009079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.009599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] PATH_INFO: `/v3/projects/48b6c97dcba94945ab5878ba4a663edf` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.011040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b6c97dcba94945ab5878ba4a663edf)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:40.013093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:40.040844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:40.074153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b6c97dcba94945ab5878ba4a663edf', 'request_id': 'req-22717b53-cbe3-4976-89e4-77b2dfdfb96b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:40.093294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b6c97dcba94945ab5878ba4a663edf', 'request_id': 'req-22717b53-cbe3-4976-89e4-77b2dfdfb96b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:40.101508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b6c97dcba94945ab5878ba4a663edf', 'request_id': 'req-22717b53-cbe3-4976-89e4-77b2dfdfb96b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:40.112640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] The token cache is being invalidate because project 48b6c97dcba94945ab5878ba4a663edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:40.112960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:40.113579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-22717b53-cbe3-4976-89e4-77b2dfdfb96b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:40.115995 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 939/1861] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:39 2022] DELETE /identity/v3/projects/48b6c97dcba94945ab5878ba4a663edf => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:40.121111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40c668-9fc7-45c4-b306-b48a6a98b822 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.121670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40c668-9fc7-45c4-b306-b48a6a98b822 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.121670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40c668-9fc7-45c4-b306-b48a6a98b822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.153280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-aa40c668-9fc7-45c4-b306-b48a6a98b822 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:40.155115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-aa40c668-9fc7-45c4-b306-b48a6a98b822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:40.234926 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 923/1862] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:40.242947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e59616-db76-45da-aebe-a57a03797120 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.243860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e59616-db76-45da-aebe-a57a03797120 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.244209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e59616-db76-45da-aebe-a57a03797120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.283308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a9e59616-db76-45da-aebe-a57a03797120 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:40.285564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a9e59616-db76-45da-aebe-a57a03797120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:40.324375 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 940/1863] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:40.329158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40c668-9fc7-45c4-b306-b48a6a98b822 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:40.342108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40c668-9fc7-45c4-b306-b48a6a98b822 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:40.347787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:40.348539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.348918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.350123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.351012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:40.351520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:40.353344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:40.368178 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 924/1864] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:40.372715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e59616-db76-45da-aebe-a57a03797120 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:40.384494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e59616-db76-45da-aebe-a57a03797120 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:40.390478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c91bf-d6cb-45a3-8298-393efcf41012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:40.391320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c91bf-d6cb-45a3-8298-393efcf41012 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.391770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c91bf-d6cb-45a3-8298-393efcf41012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.392379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c91bf-d6cb-45a3-8298-393efcf41012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.393303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c91bf-d6cb-45a3-8298-393efcf41012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:40.396626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c91bf-d6cb-45a3-8298-393efcf41012 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:40.439262 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 941/1865] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:40 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:40.444634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:40.462871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83724eb6-7bbc-4f8f-8494-95d4e82b1e13 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:40.550459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9235284-de2f-4520-a32b-7ba6ded20381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:40.552985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9235284-de2f-4520-a32b-7ba6ded20381 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.552985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9235284-de2f-4520-a32b-7ba6ded20381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.552985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9235284-de2f-4520-a32b-7ba6ded20381 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.552985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9235284-de2f-4520-a32b-7ba6ded20381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:40.556643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9235284-de2f-4520-a32b-7ba6ded20381 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:40.584767 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 925/1866] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:40 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:40.589206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c91bf-d6cb-45a3-8298-393efcf41012 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:40.602183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c91bf-d6cb-45a3-8298-393efcf41012 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:40.607560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660539f-ea88-4b35-ba4a-fe2b945788e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:40.608170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660539f-ea88-4b35-ba4a-fe2b945788e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.608456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660539f-ea88-4b35-ba4a-fe2b945788e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.608727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660539f-ea88-4b35-ba4a-fe2b945788e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.609463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660539f-ea88-4b35-ba4a-fe2b945788e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:40.613538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660539f-ea88-4b35-ba4a-fe2b945788e5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:40.628518 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 942/1867] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:40.632309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9235284-de2f-4520-a32b-7ba6ded20381 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:40.643433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9235284-de2f-4520-a32b-7ba6ded20381 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:40.650708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ee528-caaa-413e-899a-59ea602885a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:40.651490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee528-caaa-413e-899a-59ea602885a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.651907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee528-caaa-413e-899a-59ea602885a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.652281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee528-caaa-413e-899a-59ea602885a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.652950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee528-caaa-413e-899a-59ea602885a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:40.656349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee528-caaa-413e-899a-59ea602885a0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:40.678826 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 926/1868] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:40.683227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660539f-ea88-4b35-ba4a-fe2b945788e5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:40.697581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660539f-ea88-4b35-ba4a-fe2b945788e5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:40.703435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b7e3d-563d-457e-96ef-a3890d50fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:40.710897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b7e3d-563d-457e-96ef-a3890d50fecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.710897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b7e3d-563d-457e-96ef-a3890d50fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.710897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b7e3d-563d-457e-96ef-a3890d50fecc admin admin] PATH_INFO: `/v3/projects/afea87e6436644bb9cd4e4e2be7e117a/users/43acddbd8f7044c28881eabe3809b0a1/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.724652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b7e3d-563d-457e-96ef-a3890d50fecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=afea87e6436644bb9cd4e4e2be7e117a, user_id=43acddbd8f7044c28881eabe3809b0a1, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:40.727993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b7e3d-563d-457e-96ef-a3890d50fecc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:40.744283 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 943/1869] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:40 2022] PUT /identity/v3/projects/afea87e6436644bb9cd4e4e2be7e117a/users/43acddbd8f7044c28881eabe3809b0a1/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:40.754534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd09157-caa8-445c-9cb0-81655f8b2ecc None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.754810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd09157-caa8-445c-9cb0-81655f8b2ecc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.755091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd09157-caa8-445c-9cb0-81655f8b2ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.785276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8dd09157-caa8-445c-9cb0-81655f8b2ecc None None] MFA Rules not processed for user `43acddbd8f7044c28881eabe3809b0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:40.786548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8dd09157-caa8-445c-9cb0-81655f8b2ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:40.822465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:40.865484 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 927/1870] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:40.870395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b7e3d-563d-457e-96ef-a3890d50fecc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:40.934297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b7e3d-563d-457e-96ef-a3890d50fecc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:40.939414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:40.940051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:40.940342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:40.940616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:40.941259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:40.943279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:40.973876 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 944/1871] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:40 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:40.975815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd09157-caa8-445c-9cb0-81655f8b2ecc None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:40.987641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd09157-caa8-445c-9cb0-81655f8b2ecc None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.049690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eec13-c393-4995-a156-e53f06514986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.050309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eec13-c393-4995-a156-e53f06514986 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.050643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eec13-c393-4995-a156-e53f06514986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.050914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eec13-c393-4995-a156-e53f06514986 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.051541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eec13-c393-4995-a156-e53f06514986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.054399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eec13-c393-4995-a156-e53f06514986 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.062974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:41.065112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:41.079839 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 928/1872] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:40 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:41.081323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eec13-c393-4995-a156-e53f06514986 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:41.092921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eec13-c393-4995-a156-e53f06514986 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.097979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739765c-c7d5-49aa-af38-a5f91fe96137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.098622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739765c-c7d5-49aa-af38-a5f91fe96137 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.098904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739765c-c7d5-49aa-af38-a5f91fe96137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.099209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739765c-c7d5-49aa-af38-a5f91fe96137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.100516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739765c-c7d5-49aa-af38-a5f91fe96137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.102145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739765c-c7d5-49aa-af38-a5f91fe96137 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.104710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:41.112904 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 929/1873] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:41.118239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739765c-c7d5-49aa-af38-a5f91fe96137 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:41.128864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739765c-c7d5-49aa-af38-a5f91fe96137 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.134449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11772854-162e-449a-8ca7-ed500b69071e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.135104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11772854-162e-449a-8ca7-ed500b69071e admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.135388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11772854-162e-449a-8ca7-ed500b69071e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.135666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11772854-162e-449a-8ca7-ed500b69071e admin admin] PATH_INFO: `/v3/projects/d634b76f9f064a16872e6608044c512f/users/86868898575243d4a506ceef587abfce/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.136348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f7d33-3c5e-4c51-9d45-ce0a2867c934 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.140041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.140643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.140920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.141188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.142975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:41.151602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11772854-162e-449a-8ca7-ed500b69071e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d634b76f9f064a16872e6608044c512f, user_id=86868898575243d4a506ceef587abfce, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.154318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11772854-162e-449a-8ca7-ed500b69071e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.168448 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 930/1874] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:41 2022] PUT /identity/v3/projects/d634b76f9f064a16872e6608044c512f/users/86868898575243d4a506ceef587abfce/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:41.172369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11772854-162e-449a-8ca7-ed500b69071e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:41.213750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.215268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.228406 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 945/1875] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:41.251471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11772854-162e-449a-8ca7-ed500b69071e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.257650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8bd94-b57c-4476-8424-a3da08aa83fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.258372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8bd94-b57c-4476-8424-a3da08aa83fd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.258653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8bd94-b57c-4476-8424-a3da08aa83fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.258924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8bd94-b57c-4476-8424-a3da08aa83fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.259576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8bd94-b57c-4476-8424-a3da08aa83fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.261167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8bd94-b57c-4476-8424-a3da08aa83fd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.272652 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 931/1876] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:41.276927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:41.287684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ee725-5c48-4f69-b159-3302b6e5e7e1 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.292330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.292982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.293265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.293544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] PATH_INFO: `/v3/projects/d634b76f9f064a16872e6608044c512f/users/86868898575243d4a506ceef587abfce/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.295492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d634b76f9f064a16872e6608044c512f, user_id=86868898575243d4a506ceef587abfce, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.298321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.310359 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 946/1877] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:41 2022] PUT /identity/v3/projects/d634b76f9f064a16872e6608044c512f/users/86868898575243d4a506ceef587abfce/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:41.321645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768851-9c1c-419b-ab7f-48f9834a61ae None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.321942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768851-9c1c-419b-ab7f-48f9834a61ae None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.322216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65768851-9c1c-419b-ab7f-48f9834a61ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.356656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-65768851-9c1c-419b-ab7f-48f9834a61ae None None] MFA Rules not processed for user `86868898575243d4a506ceef587abfce`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:41.357998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-65768851-9c1c-419b-ab7f-48f9834a61ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:41.378369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:41.418343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:41.420257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:41.457814 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 932/1878] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:41.469828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65768851-9c1c-419b-ab7f-48f9834a61ae None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:41.512993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4b92d-8ecf-4d73-9ab6-003448b15c70 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.518003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.518667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.518953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.519229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] PATH_INFO: `/v3/users/0763c8c77a8644d1ba30aa62b4b56c9b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.520445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0763c8c77a8644d1ba30aa62b4b56c9b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.523314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.552348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65768851-9c1c-419b-ab7f-48f9834a61ae None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.557898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade1874-c624-481c-8663-2bb444ad0802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.558908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1874-c624-481c-8663-2bb444ad0802 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.559280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1874-c624-481c-8663-2bb444ad0802 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.559762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1874-c624-481c-8663-2bb444ad0802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.594325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0763c8c77a8644d1ba30aa62b4b56c9b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:41.605998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0763c8c77a8644d1ba30aa62b4b56c9b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:41.607470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0763c8c77a8644d1ba30aa62b4b56c9b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:41.608769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0763c8c77a8644d1ba30aa62b4b56c9b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:41.630184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0763c8c77a8644d1ba30aa62b4b56c9b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:41.636168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:41.646333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade1874-c624-481c-8663-2bb444ad0802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.648258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade1874-c624-481c-8663-2bb444ad0802 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.659065 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 933/1879] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:41.663684 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 947/1880] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:41 2022] DELETE /identity/v3/users/0763c8c77a8644d1ba30aa62b4b56c9b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:41.668406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0961c-c96b-4fdc-aa05-d04cef888bb0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.668770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0961c-c96b-4fdc-aa05-d04cef888bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.669108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0961c-c96b-4fdc-aa05-d04cef888bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.691040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2ed0961c-c96b-4fdc-aa05-d04cef888bb0 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:41.692322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2ed0961c-c96b-4fdc-aa05-d04cef888bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:41.818092 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 934/1881] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:41.822324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:41.824759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:41.954519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796a342-86a8-47f3-aa17-72250440486f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:41.958895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:41.959907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:41.960089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:41.960413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:41.975492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:41.977436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:41.989412 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 948/1882] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:42.306237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0961c-c96b-4fdc-aa05-d04cef888bb0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:42.308163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2ed0961c-c96b-4fdc-aa05-d04cef888bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:42.317254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:42.323043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0961c-c96b-4fdc-aa05-d04cef888bb0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:42.328095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:42.328743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:42.329018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:42.329434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] PATH_INFO: `/v3/projects/253ce34785e8458ba38c1036b7687c63` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:42.330628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253ce34785e8458ba38c1036b7687c63)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:42.332537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:42.385088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253ce34785e8458ba38c1036b7687c63', 'request_id': 'req-db573cff-8633-4e37-ad1e-1dd553de7b96'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.393419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253ce34785e8458ba38c1036b7687c63', 'request_id': 'req-db573cff-8633-4e37-ad1e-1dd553de7b96'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.394518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:42.396315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:42.410890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253ce34785e8458ba38c1036b7687c63', 'request_id': 'req-db573cff-8633-4e37-ad1e-1dd553de7b96'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.413598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2e703-1fb5-4dc5-800d-84b26be944b1 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:42.427250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] The token cache is being invalidate because project 253ce34785e8458ba38c1036b7687c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:42.428186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.429002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.431855 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 935/1883] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:42 2022] DELETE /identity/v3/projects/253ce34785e8458ba38c1036b7687c63 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:42.435785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:42.437560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:42.487412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:42.488127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:42.488516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:42.488913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] PATH_INFO: `/v3/users/f8d89ae5fffd470eaed7a3dcb09a3bd7` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:42.490599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d89ae5fffd470eaed7a3dcb09a3bd7)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:42.493103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:42.516650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db573cff-8633-4e37-ad1e-1dd553de7b96 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:42.523721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:42.524408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:42.524720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:42.525001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] PATH_INFO: `/v3/users/4ae41689ca15468e8813819a2531ff72` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:42.526354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae41689ca15468e8813819a2531ff72)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:42.528511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:42.550635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d89ae5fffd470eaed7a3dcb09a3bd7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.560052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d89ae5fffd470eaed7a3dcb09a3bd7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.560777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d89ae5fffd470eaed7a3dcb09a3bd7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.561395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d89ae5fffd470eaed7a3dcb09a3bd7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.570006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-88854648-56f9-4aa3-a272-60f12655b027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d89ae5fffd470eaed7a3dcb09a3bd7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.581826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae41689ca15468e8813819a2531ff72'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.589457 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 949/1884] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:42 2022] DELETE /identity/v3/users/f8d89ae5fffd470eaed7a3dcb09a3bd7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:42.592547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae41689ca15468e8813819a2531ff72'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.592547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae41689ca15468e8813819a2531ff72'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.592547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae41689ca15468e8813819a2531ff72'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.593984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807058ae-aa00-4c12-95ed-5757b8f91b4e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:42.594274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807058ae-aa00-4c12-95ed-5757b8f91b4e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:42.594547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807058ae-aa00-4c12-95ed-5757b8f91b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:42.598695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae41689ca15468e8813819a2531ff72'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:42.614690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-807058ae-aa00-4c12-95ed-5757b8f91b4e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:42.616234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-807058ae-aa00-4c12-95ed-5757b8f91b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:42.616838 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 936/1885] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:42 2022] DELETE /identity/v3/users/4ae41689ca15468e8813819a2531ff72 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:42.620570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:42.680681 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 950/1886] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:42.691256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:42.692754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:42.703817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:42.756638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb833a3-b66c-4469-8f4d-a802743bb089 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:42.760174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:42.760799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:42.761061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:42.761376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:42.773865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:42.775297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:42.786816 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 937/1887] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:43.117911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807058ae-aa00-4c12-95ed-5757b8f91b4e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:43.119387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-807058ae-aa00-4c12-95ed-5757b8f91b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:43.139842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807058ae-aa00-4c12-95ed-5757b8f91b4e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:43.171930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:43.172865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.173204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.173478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] PATH_INFO: `/v3/projects/b6f449bc790d4b7b95d314de67e10a75` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.176255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f449bc790d4b7b95d314de67e10a75)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:43.178039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:43.283679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f449bc790d4b7b95d314de67e10a75', 'request_id': 'req-4693a751-f91e-419c-852a-6f8974e26d10'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.313196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f449bc790d4b7b95d314de67e10a75', 'request_id': 'req-4693a751-f91e-419c-852a-6f8974e26d10'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.323481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:43.325594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:43.328296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f449bc790d4b7b95d314de67e10a75', 'request_id': 'req-4693a751-f91e-419c-852a-6f8974e26d10'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.345131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] The token cache is being invalidate because project b6f449bc790d4b7b95d314de67e10a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:43.345427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.346643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4693a751-f91e-419c-852a-6f8974e26d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.348530 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 951/1888] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:43 2022] DELETE /identity/v3/projects/b6f449bc790d4b7b95d314de67e10a75 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:43.357533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb7aab-6949-4aed-901c-026302779d1a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.357533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb7aab-6949-4aed-901c-026302779d1a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.357533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb7aab-6949-4aed-901c-026302779d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.380052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-55eb7aab-6949-4aed-901c-026302779d1a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:43.386354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-55eb7aab-6949-4aed-901c-026302779d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:43.435319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf022ac-7c56-4095-aca2-3edf164a4f5f service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:43.475993 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 952/1889] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:43.475993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb7aab-6949-4aed-901c-026302779d1a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:43.475993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-55eb7aab-6949-4aed-901c-026302779d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:43.483528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb7aab-6949-4aed-901c-026302779d1a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:43.488325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:43.488943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.489219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.489489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] PATH_INFO: `/v3/projects/7c2d8e595a7d4edba9d798b892dfc32f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.491317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c2d8e595a7d4edba9d798b892dfc32f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:43.493222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:43.504442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:43.505061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.505369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.506322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.508145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:43.525649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:43.542993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c2d8e595a7d4edba9d798b892dfc32f', 'request_id': 'req-8c196210-1a5a-4cda-918e-b252f5d39eba'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.550481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c2d8e595a7d4edba9d798b892dfc32f', 'request_id': 'req-8c196210-1a5a-4cda-918e-b252f5d39eba'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.557530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c2d8e595a7d4edba9d798b892dfc32f', 'request_id': 'req-8c196210-1a5a-4cda-918e-b252f5d39eba'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.568640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] The token cache is being invalidate because project 7c2d8e595a7d4edba9d798b892dfc32f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:43.569078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.569799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c196210-1a5a-4cda-918e-b252f5d39eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.571484 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 953/1890] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:43 2022] DELETE /identity/v3/projects/7c2d8e595a7d4edba9d798b892dfc32f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:43.574495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e10534-5338-41e7-a794-505aafd0fe26 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.574495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e10534-5338-41e7-a794-505aafd0fe26 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.574686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:43.574779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e10534-5338-41e7-a794-505aafd0fe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.576118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:43.586837 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 938/1891] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:43.587809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:43.595640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c1e10534-5338-41e7-a794-505aafd0fe26 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:43.596931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c1e10534-5338-41e7-a794-505aafd0fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:43.600815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36273aa-6946-4e34-863c-a0b94719222b service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:43.617361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:43.671824 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 954/1892] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:43.676315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e10534-5338-41e7-a794-505aafd0fe26 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:43.683600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:43.685817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.686561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.686927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] PATH_INFO: `/v3/users/f131966abe52457a9a64ae0faba9b628` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.687935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e10534-5338-41e7-a794-505aafd0fe26 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:43.688529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f131966abe52457a9a64ae0faba9b628)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:43.690857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:43.693690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:43.694899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.695258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.695751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.696519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:43.696889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:43.699258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:43.714824 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 955/1893] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:36:43.716258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:43.727490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e378ee-c2c4-4bb7-b4f1-03143049b14c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:43.737217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:43.737959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.738249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.738555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.742287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:43.744588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:43.766200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f131966abe52457a9a64ae0faba9b628'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.775102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f131966abe52457a9a64ae0faba9b628'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.775904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f131966abe52457a9a64ae0faba9b628'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.776497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f131966abe52457a9a64ae0faba9b628'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.781679 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 956/1894] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:43 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:43.785950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:43.789272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:43.790229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f131966abe52457a9a64ae0faba9b628'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:43.823860 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 939/1895] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:43 2022] DELETE /identity/v3/users/f131966abe52457a9a64ae0faba9b628 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:43.882110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278bd33-7f85-4eeb-a21a-0844a8b8193f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:43.912191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:43.953995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfddb67-5273-4d82-9d8b-9a4034508251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:43.954888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfddb67-5273-4d82-9d8b-9a4034508251 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:43.955316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfddb67-5273-4d82-9d8b-9a4034508251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:43.955736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfddb67-5273-4d82-9d8b-9a4034508251 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:43.956578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfddb67-5273-4d82-9d8b-9a4034508251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:43.959148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfddb67-5273-4d82-9d8b-9a4034508251 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:43.985970 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 957/1896] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:43 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:43.990803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.004657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7941ff-147c-4e46-9181-97b4634a7ee9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.010223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4591e-1a6a-43c0-b998-7ecd01748198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.010852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4591e-1a6a-43c0-b998-7ecd01748198 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.011156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4591e-1a6a-43c0-b998-7ecd01748198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.011599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4591e-1a6a-43c0-b998-7ecd01748198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.012423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4591e-1a6a-43c0-b998-7ecd01748198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.014482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4591e-1a6a-43c0-b998-7ecd01748198 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.028671 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 940/1897] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:44.033604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfddb67-5273-4d82-9d8b-9a4034508251 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.045443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfddb67-5273-4d82-9d8b-9a4034508251 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.050241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.050863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.051144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.051428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.052086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.053850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.064353 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 958/1898] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:44.069090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4591e-1a6a-43c0-b998-7ecd01748198 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.080983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4591e-1a6a-43c0-b998-7ecd01748198 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.090010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c38170-b14a-4df8-aa13-dc56a29b600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.090748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c38170-b14a-4df8-aa13-dc56a29b600f admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.091487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c38170-b14a-4df8-aa13-dc56a29b600f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.091487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c38170-b14a-4df8-aa13-dc56a29b600f admin admin] PATH_INFO: `/v3/projects/bedb38be82b44b54b6b3efe7e4b99597/users/1ee383cc36cc40258d72f77d8e5560aa/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.111121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c38170-b14a-4df8-aa13-dc56a29b600f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedb38be82b44b54b6b3efe7e4b99597, user_id=1ee383cc36cc40258d72f77d8e5560aa, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.115152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c38170-b14a-4df8-aa13-dc56a29b600f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.133050 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 941/1899] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:44 2022] PUT /identity/v3/projects/bedb38be82b44b54b6b3efe7e4b99597/users/1ee383cc36cc40258d72f77d8e5560aa/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:44.143038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.143835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaf372-cd47-44b8-bcd4-49e2ccc612b3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.144316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaf372-cd47-44b8-bcd4-49e2ccc612b3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.144589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaf372-cd47-44b8-bcd4-49e2ccc612b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.145010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:44.159780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6852ff-fb43-469c-9da7-51364c1f5321 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.189185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e6aaf372-cd47-44b8-bcd4-49e2ccc612b3 None None] MFA Rules not processed for user `1ee383cc36cc40258d72f77d8e5560aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:44.190014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e6aaf372-cd47-44b8-bcd4-49e2ccc612b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:44.234367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.235610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.236085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.236643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] PATH_INFO: `/v3/projects/574f5dfe9f4344bfabe064c2367098cd` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.238074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=574f5dfe9f4344bfabe064c2367098cd)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.240708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.256464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:44.283807 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 942/1900] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:44.299623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaf372-cd47-44b8-bcd4-49e2ccc612b3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.327799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574f5dfe9f4344bfabe064c2367098cd', 'request_id': 'req-3a6cf450-a121-4058-80a2-d32dfa2ee880'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:44.331616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaf372-cd47-44b8-bcd4-49e2ccc612b3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.344583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574f5dfe9f4344bfabe064c2367098cd', 'request_id': 'req-3a6cf450-a121-4058-80a2-d32dfa2ee880'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:44.355306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574f5dfe9f4344bfabe064c2367098cd', 'request_id': 'req-3a6cf450-a121-4058-80a2-d32dfa2ee880'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:44.382943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] The token cache is being invalidate because project 574f5dfe9f4344bfabe064c2367098cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:44.383260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:44.384638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a6cf450-a121-4058-80a2-d32dfa2ee880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:44.386574 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 959/1901] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:44 2022] DELETE /identity/v3/projects/574f5dfe9f4344bfabe064c2367098cd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:44.392795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995462f3-b562-46fe-9602-9ee9fc4bbb90 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.393063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995462f3-b562-46fe-9602-9ee9fc4bbb90 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.393404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995462f3-b562-46fe-9602-9ee9fc4bbb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.437577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-995462f3-b562-46fe-9602-9ee9fc4bbb90 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:44.439109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-995462f3-b562-46fe-9602-9ee9fc4bbb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:44.489826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:44.510429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.511062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.511343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.511623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.512274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.515205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.550266 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 960/1902] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:44.560314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907317f4-9aa8-466e-91e6-7f2c3c511655 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.560839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907317f4-9aa8-466e-91e6-7f2c3c511655 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.561696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907317f4-9aa8-466e-91e6-7f2c3c511655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.574894 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 943/1903] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:44 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:44.580421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.583476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:44.593997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-907317f4-9aa8-466e-91e6-7f2c3c511655 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:44.595231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-907317f4-9aa8-466e-91e6-7f2c3c511655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:44.702441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b35a6-263f-408a-ae1d-994cefe0b161 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.710841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce92054-24df-4e64-a321-8edc29f3d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.711564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce92054-24df-4e64-a321-8edc29f3d6af admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.711932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce92054-24df-4e64-a321-8edc29f3d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.712609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce92054-24df-4e64-a321-8edc29f3d6af admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.713412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce92054-24df-4e64-a321-8edc29f3d6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.730086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce92054-24df-4e64-a321-8edc29f3d6af admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.749176 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 961/1904] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:44.753513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907317f4-9aa8-466e-91e6-7f2c3c511655 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.774229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907317f4-9aa8-466e-91e6-7f2c3c511655 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.789289 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 944/1905] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:44 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:44.805380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce92054-24df-4e64-a321-8edc29f3d6af admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.809232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.810092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.811555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.812609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.814377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:44.814862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.820583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.828499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce92054-24df-4e64-a321-8edc29f3d6af admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.842762 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 962/1906] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:36:44.845800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3d4f-db7c-4892-a90d-395a3b2380e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.846870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3d4f-db7c-4892-a90d-395a3b2380e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.847982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3d4f-db7c-4892-a90d-395a3b2380e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.848238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3d4f-db7c-4892-a90d-395a3b2380e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.848771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3d4f-db7c-4892-a90d-395a3b2380e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.851921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.852825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3d4f-db7c-4892-a90d-395a3b2380e8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.889931 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 945/1907] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:44.893809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3d4f-db7c-4892-a90d-395a3b2380e8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:44.899259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b02297-0847-4464-af9f-dd73227cafd5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.910205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3d4f-db7c-4892-a90d-395a3b2380e8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:44.917686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4554280-5b67-4ad8-9187-4df153adf4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.919230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b340d-1760-4509-8e06-049b2a202725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:44.919744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554280-5b67-4ad8-9187-4df153adf4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.920635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b340d-1760-4509-8e06-049b2a202725 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:44.920918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b340d-1760-4509-8e06-049b2a202725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.921146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554280-5b67-4ad8-9187-4df153adf4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:44.921256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b340d-1760-4509-8e06-049b2a202725 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.921558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554280-5b67-4ad8-9187-4df153adf4b8 admin admin] PATH_INFO: `/v3/projects/67d09a67d8a941478d9b545177a141d9/users/484549bebc884fad838e5aef43df0159/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:44.921858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b340d-1760-4509-8e06-049b2a202725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.929758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b340d-1760-4509-8e06-049b2a202725 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.945875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4554280-5b67-4ad8-9187-4df153adf4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d09a67d8a941478d9b545177a141d9, user_id=484549bebc884fad838e5aef43df0159, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:44.951175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4554280-5b67-4ad8-9187-4df153adf4b8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:44.978431 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 946/1908] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:44 2022] PUT /identity/v3/projects/67d09a67d8a941478d9b545177a141d9/users/484549bebc884fad838e5aef43df0159/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:44.982707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4554280-5b67-4ad8-9187-4df153adf4b8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.002318 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 963/1909] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:44 2022] POST /identity/v3/projects => generated 399 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:45.007807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b340d-1760-4509-8e06-049b2a202725 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.029161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:45.081932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4554280-5b67-4ad8-9187-4df153adf4b8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.122637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b340d-1760-4509-8e06-049b2a202725 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.127338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340509-0ea2-4bc9-b23c-4f7561749821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.127946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340509-0ea2-4bc9-b23c-4f7561749821 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.128601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340509-0ea2-4bc9-b23c-4f7561749821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.128601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9340509-0ea2-4bc9-b23c-4f7561749821 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.129118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9340509-0ea2-4bc9-b23c-4f7561749821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.131859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9340509-0ea2-4bc9-b23c-4f7561749821 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.151915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d1319-290c-4aa1-b5f3-8b208c882b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.152763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d1319-290c-4aa1-b5f3-8b208c882b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.153147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d1319-290c-4aa1-b5f3-8b208c882b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.153693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d1319-290c-4aa1-b5f3-8b208c882b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.154316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d1319-290c-4aa1-b5f3-8b208c882b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.156022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d1319-290c-4aa1-b5f3-8b208c882b64 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.157283 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 964/1910] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:45 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:45.161367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340509-0ea2-4bc9-b23c-4f7561749821 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.172711 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 947/1911] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:45.172855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9340509-0ea2-4bc9-b23c-4f7561749821 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.173739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d1319-290c-4aa1-b5f3-8b208c882b64 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.178002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e796-b90f-475e-bd2d-0d741fb42c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.178729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e796-b90f-475e-bd2d-0d741fb42c91 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.179007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e796-b90f-475e-bd2d-0d741fb42c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.179574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e796-b90f-475e-bd2d-0d741fb42c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.180248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917e796-b90f-475e-bd2d-0d741fb42c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.182482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917e796-b90f-475e-bd2d-0d741fb42c91 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.187074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d1319-290c-4aa1-b5f3-8b208c882b64 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.193648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df27b8-391e-4681-803b-5e77451461f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.194615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df27b8-391e-4681-803b-5e77451461f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.194734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df27b8-391e-4681-803b-5e77451461f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.195007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df27b8-391e-4681-803b-5e77451461f3 admin admin] PATH_INFO: `/v3/projects/67d09a67d8a941478d9b545177a141d9/users/484549bebc884fad838e5aef43df0159/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.195101 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 965/1912] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:45.196995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df27b8-391e-4681-803b-5e77451461f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d09a67d8a941478d9b545177a141d9, user_id=484549bebc884fad838e5aef43df0159, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.199394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e796-b90f-475e-bd2d-0d741fb42c91 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.199708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df27b8-391e-4681-803b-5e77451461f3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.211041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e796-b90f-475e-bd2d-0d741fb42c91 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.212119 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 948/1913] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:45 2022] PUT /identity/v3/projects/67d09a67d8a941478d9b545177a141d9/users/484549bebc884fad838e5aef43df0159/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:45.222282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8755c-310c-4efb-8677-bac4372a8db4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.223173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8755c-310c-4efb-8677-bac4372a8db4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.223554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8755c-310c-4efb-8677-bac4372a8db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.237214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:45.263674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-45d8755c-310c-4efb-8677-bac4372a8db4 None None] MFA Rules not processed for user `484549bebc884fad838e5aef43df0159`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:45.265365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-45d8755c-310c-4efb-8677-bac4372a8db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:45.288662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36180e-bb31-453a-89bf-1f7b05307e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.289590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36180e-bb31-453a-89bf-1f7b05307e1e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.289896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36180e-bb31-453a-89bf-1f7b05307e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.290175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36180e-bb31-453a-89bf-1f7b05307e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.290872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36180e-bb31-453a-89bf-1f7b05307e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.292593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36180e-bb31-453a-89bf-1f7b05307e1e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.306346 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 966/1914] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:45.311519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36180e-bb31-453a-89bf-1f7b05307e1e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.326613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36180e-bb31-453a-89bf-1f7b05307e1e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.332410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd7a7-bf80-44d1-b137-55bb5987ea84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.333116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd7a7-bf80-44d1-b137-55bb5987ea84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.333400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd7a7-bf80-44d1-b137-55bb5987ea84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.334348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd7a7-bf80-44d1-b137-55bb5987ea84 admin admin] PATH_INFO: `/v3/projects/40c8bb7495454e54beaaa5ff7d612adb/users/ac9340771ae345a7957274f0043d0586/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.355097 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 949/1915] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:45.363550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcd7a7-bf80-44d1-b137-55bb5987ea84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c8bb7495454e54beaaa5ff7d612adb, user_id=ac9340771ae345a7957274f0043d0586, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.367482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8755c-310c-4efb-8677-bac4372a8db4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.368093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcd7a7-bf80-44d1-b137-55bb5987ea84 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.368913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-45d8755c-310c-4efb-8677-bac4372a8db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:45.392453 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 967/1916] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:45 2022] PUT /identity/v3/projects/40c8bb7495454e54beaaa5ff7d612adb/users/ac9340771ae345a7957274f0043d0586/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:45.403515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742817b-5ce4-4928-b5a2-0dda7b3832c8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.403852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742817b-5ce4-4928-b5a2-0dda7b3832c8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.404133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742817b-5ce4-4928-b5a2-0dda7b3832c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.430550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:45.434943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1742817b-5ce4-4928-b5a2-0dda7b3832c8 None None] MFA Rules not processed for user `ac9340771ae345a7957274f0043d0586`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:45.436290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1742817b-5ce4-4928-b5a2-0dda7b3832c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:45.442634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8755c-310c-4efb-8677-bac4372a8db4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.493458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b9ad6-4cec-42e6-a839-f6e45fafdfe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.494113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b9ad6-4cec-42e6-a839-f6e45fafdfe4 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.494397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b9ad6-4cec-42e6-a839-f6e45fafdfe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.494671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b9ad6-4cec-42e6-a839-f6e45fafdfe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.501070 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 968/1917] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:45.506171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742817b-5ce4-4928-b5a2-0dda7b3832c8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.558035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b9ad6-4cec-42e6-a839-f6e45fafdfe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.559515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b9ad6-4cec-42e6-a839-f6e45fafdfe4 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.571443 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 950/1918] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:45.581965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742817b-5ce4-4928-b5a2-0dda7b3832c8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.586969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad23de-3703-4884-a3cb-90febaf3462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.587366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad23de-3703-4884-a3cb-90febaf3462b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.587778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad23de-3703-4884-a3cb-90febaf3462b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.588184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad23de-3703-4884-a3cb-90febaf3462b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.588907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad23de-3703-4884-a3cb-90febaf3462b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.591035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad23de-3703-4884-a3cb-90febaf3462b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.633436 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 969/1919] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:45 2022] POST /identity/v3/projects => generated 401 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:45.635846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b9ad6-4cec-42e6-a839-f6e45fafdfe4 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.647719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b9ad6-4cec-42e6-a839-f6e45fafdfe4 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.685023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad23de-3703-4884-a3cb-90febaf3462b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.697336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:45.721487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aeff8-28cf-4172-9930-4407d3a0d893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.722747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aeff8-28cf-4172-9930-4407d3a0d893 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.723192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aeff8-28cf-4172-9930-4407d3a0d893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.723476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aeff8-28cf-4172-9930-4407d3a0d893 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.724640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25aeff8-28cf-4172-9930-4407d3a0d893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.726654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25aeff8-28cf-4172-9930-4407d3a0d893 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.750286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad23de-3703-4884-a3cb-90febaf3462b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.752988 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 951/1920] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:45 2022] POST /identity/v3/users => generated 307 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:45.756491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aeff8-28cf-4172-9930-4407d3a0d893 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.757982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.759045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.759174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.759529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.761433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:45.767235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aeff8-28cf-4172-9930-4407d3a0d893 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.773807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e4815-587d-4f8a-8be9-116763d53afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.774467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e4815-587d-4f8a-8be9-116763d53afb admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.774750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e4815-587d-4f8a-8be9-116763d53afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.775026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e4815-587d-4f8a-8be9-116763d53afb admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.775679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e4815-587d-4f8a-8be9-116763d53afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.777306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e4815-587d-4f8a-8be9-116763d53afb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.788517 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 952/1921] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:45.793367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e4815-587d-4f8a-8be9-116763d53afb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.803452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e4815-587d-4f8a-8be9-116763d53afb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.808232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b9f0e-c460-46e7-acf7-c3d3ad39baa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.808899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b9f0e-c460-46e7-acf7-c3d3ad39baa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.809225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b9f0e-c460-46e7-acf7-c3d3ad39baa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.810087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b9f0e-c460-46e7-acf7-c3d3ad39baa7 admin admin] PATH_INFO: `/v3/projects/2d9fb779c61c472e9c2ab1758695e705/users/c2eb24edb4a3424b954ae59a7e5678a8/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.832894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.835406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.837191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b9f0e-c460-46e7-acf7-c3d3ad39baa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9fb779c61c472e9c2ab1758695e705, user_id=c2eb24edb4a3424b954ae59a7e5678a8, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.840959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b9f0e-c460-46e7-acf7-c3d3ad39baa7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.848985 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 970/1922] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:45.866853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 953/1923] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:45 2022] PUT /identity/v3/projects/2d9fb779c61c472e9c2ab1758695e705/users/c2eb24edb4a3424b954ae59a7e5678a8/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:45.871063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:45.885465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99e34e-013a-44cf-a342-cf478f55c419 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:45.966949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e30ab0-625f-408a-a42b-0b04d2356ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:45.967444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e30ab0-625f-408a-a42b-0b04d2356ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:45.968768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e30ab0-625f-408a-a42b-0b04d2356ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:45.969051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e30ab0-625f-408a-a42b-0b04d2356ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:45.970503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e30ab0-625f-408a-a42b-0b04d2356ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:45.972172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e30ab0-625f-408a-a42b-0b04d2356ab0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:45.987650 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 971/1924] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:45.994860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b9f0e-c460-46e7-acf7-c3d3ad39baa7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:46.004259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:46.010243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b9f0e-c460-46e7-acf7-c3d3ad39baa7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:46.022903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:46.022903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:46.023100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:46.024124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] PATH_INFO: `/v3/projects/2d9fb779c61c472e9c2ab1758695e705/users/c2eb24edb4a3424b954ae59a7e5678a8/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:46.027541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9fb779c61c472e9c2ab1758695e705, user_id=c2eb24edb4a3424b954ae59a7e5678a8, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:46.031556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:46.044899 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 954/1925] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:45 2022] PUT /identity/v3/projects/2d9fb779c61c472e9c2ab1758695e705/users/c2eb24edb4a3424b954ae59a7e5678a8/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:46.055868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e344a8-4e5d-49d2-94a0-ec844bd7a829 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:46.056162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e344a8-4e5d-49d2-94a0-ec844bd7a829 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:46.056525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e344a8-4e5d-49d2-94a0-ec844bd7a829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:46.092664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d1e344a8-4e5d-49d2-94a0-ec844bd7a829 None None] MFA Rules not processed for user `c2eb24edb4a3424b954ae59a7e5678a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:46.094485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d1e344a8-4e5d-49d2-94a0-ec844bd7a829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:46.175577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:46.207131 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 972/1926] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:46 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:46.219894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:46.221592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:46.301431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562dce0-e6d3-484c-b946-67142d747d67 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:46.312597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d41df-43b2-4d56-9aa7-e5389ee8e644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:46.313207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d41df-43b2-4d56-9aa7-e5389ee8e644 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:46.313663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d41df-43b2-4d56-9aa7-e5389ee8e644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:46.313929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d41df-43b2-4d56-9aa7-e5389ee8e644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:46.329208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d41df-43b2-4d56-9aa7-e5389ee8e644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:46.330892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d41df-43b2-4d56-9aa7-e5389ee8e644 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:46.342365 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 955/1927] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:46 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:46.428166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e344a8-4e5d-49d2-94a0-ec844bd7a829 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:46.437958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e344a8-4e5d-49d2-94a0-ec844bd7a829 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:46.441495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85940f65-6721-4f28-bacf-ea87c89c9945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:46.442122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85940f65-6721-4f28-bacf-ea87c89c9945 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:46.442401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85940f65-6721-4f28-bacf-ea87c89c9945 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:46.442667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85940f65-6721-4f28-bacf-ea87c89c9945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:46.503557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85940f65-6721-4f28-bacf-ea87c89c9945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:46.505037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85940f65-6721-4f28-bacf-ea87c89c9945 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:46.515421 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 973/1928] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:47.048043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0e99b-7725-456b-9782-63379019ef3a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:47.048507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0e99b-7725-456b-9782-63379019ef3a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:47.048584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0e99b-7725-456b-9782-63379019ef3a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:47.050432 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 956/1929] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:36:47.064725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18597d5-ebc7-4d7b-802b-b6e3d9d5676e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:47.065564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18597d5-ebc7-4d7b-802b-b6e3d9d5676e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:47.065957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18597d5-ebc7-4d7b-802b-b6e3d9d5676e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:47.088403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b18597d5-ebc7-4d7b-802b-b6e3d9d5676e None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:47.089659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b18597d5-ebc7-4d7b-802b-b6e3d9d5676e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:47.230782 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 974/1930] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:47.508672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0e99b-7725-456b-9782-63379019ef3a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:47.511328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-47c0e99b-7725-456b-9782-63379019ef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:47.547054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0e99b-7725-456b-9782-63379019ef3a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:47.557538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0e983-9d3e-4e0f-b480-32e19293a5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:47.558792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0e983-9d3e-4e0f-b480-32e19293a5c0 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:47.559229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0e983-9d3e-4e0f-b480-32e19293a5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:47.561952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0e983-9d3e-4e0f-b480-32e19293a5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:47.582944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0e983-9d3e-4e0f-b480-32e19293a5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:47.587515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0e983-9d3e-4e0f-b480-32e19293a5c0 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:47.632298 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 957/1931] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:47.761086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18597d5-ebc7-4d7b-802b-b6e3d9d5676e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:47.763305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b18597d5-ebc7-4d7b-802b-b6e3d9d5676e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:47.938019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18597d5-ebc7-4d7b-802b-b6e3d9d5676e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:47.946136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:47.948120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:47.948627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:47.949124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] PATH_INFO: `/v3/users/43acddbd8f7044c28881eabe3809b0a1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:47.951197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43acddbd8f7044c28881eabe3809b0a1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:47.955168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:47.974123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:48.015270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43acddbd8f7044c28881eabe3809b0a1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.024752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43acddbd8f7044c28881eabe3809b0a1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.025549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43acddbd8f7044c28881eabe3809b0a1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.026609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43acddbd8f7044c28881eabe3809b0a1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.035961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43acddbd8f7044c28881eabe3809b0a1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.056556 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 975/1932] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:47 2022] DELETE /identity/v3/users/43acddbd8f7044c28881eabe3809b0a1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:48.061706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb549cc-ea01-4da1-bec8-174243fab86f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:48.062254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb549cc-ea01-4da1-bec8-174243fab86f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:48.062254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb549cc-ea01-4da1-bec8-174243fab86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:48.084203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7fb549cc-ea01-4da1-bec8-174243fab86f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:48.085723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7fb549cc-ea01-4da1-bec8-174243fab86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:48.165747 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 958/1933] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:48.178188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:48.179685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:48.265437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b26fd5-6f63-4fbb-ae07-d091d694d203 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:48.269272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56853c0-37e7-412f-876d-4dd1d2db107f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:48.269916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56853c0-37e7-412f-876d-4dd1d2db107f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:48.270190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56853c0-37e7-412f-876d-4dd1d2db107f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:48.270459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56853c0-37e7-412f-876d-4dd1d2db107f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:48.280845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56853c0-37e7-412f-876d-4dd1d2db107f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:48.282311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56853c0-37e7-412f-876d-4dd1d2db107f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:48.292316 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 976/1934] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:48.569003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb549cc-ea01-4da1-bec8-174243fab86f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:48.570636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7fb549cc-ea01-4da1-bec8-174243fab86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:48.588671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb549cc-ea01-4da1-bec8-174243fab86f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:48.605355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:48.606061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:48.606488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:48.606770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] PATH_INFO: `/v3/projects/afea87e6436644bb9cd4e4e2be7e117a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:48.607965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afea87e6436644bb9cd4e4e2be7e117a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:48.609925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:48.696375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afea87e6436644bb9cd4e4e2be7e117a', 'request_id': 'req-7064c57e-87cc-4aae-bf58-959ecdcd2415'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.706742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afea87e6436644bb9cd4e4e2be7e117a', 'request_id': 'req-7064c57e-87cc-4aae-bf58-959ecdcd2415'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.715898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afea87e6436644bb9cd4e4e2be7e117a', 'request_id': 'req-7064c57e-87cc-4aae-bf58-959ecdcd2415'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.730816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] The token cache is being invalidate because project afea87e6436644bb9cd4e4e2be7e117a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:48.731222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.731851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:48.736490 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 959/1935] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:48 2022] DELETE /identity/v3/projects/afea87e6436644bb9cd4e4e2be7e117a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:48.738460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56853c0-37e7-412f-876d-4dd1d2db107f service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:48.753961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56853c0-37e7-412f-876d-4dd1d2db107f service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:48.870774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:48.871580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:48.871868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:48.872147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] PATH_INFO: `/v3/users/86868898575243d4a506ceef587abfce` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:48.887800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=86868898575243d4a506ceef587abfce)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:48.890158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:48.933438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:48.937137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:48.954066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:48.964070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c57e-87cc-4aae-bf58-959ecdcd2415 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:48.980585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:48.981621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:48.982068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:48.982475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] PATH_INFO: `/v3/users/1ee383cc36cc40258d72f77d8e5560aa` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:48.984145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee383cc36cc40258d72f77d8e5560aa)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:48.987552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:49.004346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:49.016005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86868898575243d4a506ceef587abfce'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.028833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86868898575243d4a506ceef587abfce'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.029903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86868898575243d4a506ceef587abfce'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.030619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86868898575243d4a506ceef587abfce'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.044282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ed8ac39c-77a4-4c13-8da8-f9695185d61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86868898575243d4a506ceef587abfce'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.086006 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 977/1936] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:48 2022] DELETE /identity/v3/users/86868898575243d4a506ceef587abfce => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:49.122431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee383cc36cc40258d72f77d8e5560aa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.144171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee383cc36cc40258d72f77d8e5560aa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.144967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee383cc36cc40258d72f77d8e5560aa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.146512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee383cc36cc40258d72f77d8e5560aa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.156966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee383cc36cc40258d72f77d8e5560aa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.197508 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 960/1937] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:48 2022] DELETE /identity/v3/users/1ee383cc36cc40258d72f77d8e5560aa => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:49.203054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7c527-7ff6-4d35-9af3-4ef079a5e713 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:49.203634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7c527-7ff6-4d35-9af3-4ef079a5e713 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:49.204327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7c527-7ff6-4d35-9af3-4ef079a5e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:49.229667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5da7c527-7ff6-4d35-9af3-4ef079a5e713 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:49.231960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5da7c527-7ff6-4d35-9af3-4ef079a5e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:49.330881 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 978/1938] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:49.339621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:49.340960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:49.456907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f667287-6b35-4b9e-b0ef-5fe86daf52ed admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:49.466153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab9dc2-52ef-4164-83c7-7720e0642cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:49.469367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab9dc2-52ef-4164-83c7-7720e0642cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:49.469669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab9dc2-52ef-4164-83c7-7720e0642cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:49.470191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab9dc2-52ef-4164-83c7-7720e0642cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:49.482758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7c527-7ff6-4d35-9af3-4ef079a5e713 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:49.485087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5da7c527-7ff6-4d35-9af3-4ef079a5e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:49.508599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7c527-7ff6-4d35-9af3-4ef079a5e713 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:49.509560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab9dc2-52ef-4164-83c7-7720e0642cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:49.513017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab9dc2-52ef-4164-83c7-7720e0642cfb service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:49.523437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:49.524164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:49.524450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:49.524798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] PATH_INFO: `/v3/projects/d634b76f9f064a16872e6608044c512f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:49.526128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d634b76f9f064a16872e6608044c512f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:49.528316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:49.531174 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 961/1939] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:49.593984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd634b76f9f064a16872e6608044c512f', 'request_id': 'req-f731cbc7-e31c-4303-9615-d529a9664451'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.606878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd634b76f9f064a16872e6608044c512f', 'request_id': 'req-f731cbc7-e31c-4303-9615-d529a9664451'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.617234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd634b76f9f064a16872e6608044c512f', 'request_id': 'req-f731cbc7-e31c-4303-9615-d529a9664451'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.630341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] The token cache is being invalidate because project d634b76f9f064a16872e6608044c512f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:49.630733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.631553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f731cbc7-e31c-4303-9615-d529a9664451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:49.637864 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 979/1940] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:49 2022] DELETE /identity/v3/projects/d634b76f9f064a16872e6608044c512f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:49.639341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a764e6-e81c-46e8-85be-d4a37989007f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:49.639648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a764e6-e81c-46e8-85be-d4a37989007f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:49.639923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a764e6-e81c-46e8-85be-d4a37989007f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:49.664967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-84a764e6-e81c-46e8-85be-d4a37989007f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:49.666608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-84a764e6-e81c-46e8-85be-d4a37989007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:49.757434 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 962/1941] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:49.765493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15341bee-5a16-46f5-a6eb-4107c894bda8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:49.765831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15341bee-5a16-46f5-a6eb-4107c894bda8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:49.766118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15341bee-5a16-46f5-a6eb-4107c894bda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:49.790773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-15341bee-5a16-46f5-a6eb-4107c894bda8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:49.792382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-15341bee-5a16-46f5-a6eb-4107c894bda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:49.806290 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 980/1942] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:49.810367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a764e6-e81c-46e8-85be-d4a37989007f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:49.828888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a764e6-e81c-46e8-85be-d4a37989007f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:49.834481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:49.835321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:49.835706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:49.836364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:49.837520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:49.837520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:49.839397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:49.853124 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 963/1943] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:49.857876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15341bee-5a16-46f5-a6eb-4107c894bda8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:49.868804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15341bee-5a16-46f5-a6eb-4107c894bda8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:49.874434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89069af-b14f-47d8-9fd3-a4b18336dd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:49.876035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89069af-b14f-47d8-9fd3-a4b18336dd3a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:49.876556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89069af-b14f-47d8-9fd3-a4b18336dd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:49.877032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89069af-b14f-47d8-9fd3-a4b18336dd3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:49.877992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89069af-b14f-47d8-9fd3-a4b18336dd3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:49.881043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89069af-b14f-47d8-9fd3-a4b18336dd3a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:49.884326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:49.886498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:49.910901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340a02-fc0f-4991-8d3f-d67f86006e3a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:49.919274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:49.920536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:49.921192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:49.921618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] PATH_INFO: `/v3/projects/bedb38be82b44b54b6b3efe7e4b99597` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:49.923376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bedb38be82b44b54b6b3efe7e4b99597)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:49.940242 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 981/1944] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:49 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:49.940486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:49.943259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:49.947179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89069af-b14f-47d8-9fd3-a4b18336dd3a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:49.979158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:50.088692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bedb38be82b44b54b6b3efe7e4b99597', 'request_id': 'req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.105395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bedb38be82b44b54b6b3efe7e4b99597', 'request_id': 'req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.109390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89069af-b14f-47d8-9fd3-a4b18336dd3a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.138953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bedb38be82b44b54b6b3efe7e4b99597', 'request_id': 'req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.156152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] The token cache is being invalidate because project bedb38be82b44b54b6b3efe7e4b99597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:50.156562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.157358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.160045 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 964/1945] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:49 2022] DELETE /identity/v3/projects/bedb38be82b44b54b6b3efe7e4b99597 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:50.166118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.167181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:50.177858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.178612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.179100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.179444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.180611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.182950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.240707 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 982/1946] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:49 2022] POST /identity/v3/users => generated 292 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:50.247636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.247636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:50.279417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b52b0-6b6d-46e6-b1b7-e840b07c14a9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.296874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89288b8b-9fcc-47d5-805d-43ae5ef3c127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.298321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288b8b-9fcc-47d5-805d-43ae5ef3c127 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.298863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288b8b-9fcc-47d5-805d-43ae5ef3c127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.299427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89288b8b-9fcc-47d5-805d-43ae5ef3c127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.300773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89288b8b-9fcc-47d5-805d-43ae5ef3c127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.303101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89288b8b-9fcc-47d5-805d-43ae5ef3c127 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.318337 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 983/1947] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:50.319662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89288b8b-9fcc-47d5-805d-43ae5ef3c127 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.320782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8bae-a644-4f6f-9bfb-136d9060eea5 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.326908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.327625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.327907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.328193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] PATH_INFO: `/v3/users/484549bebc884fad838e5aef43df0159` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.329545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=484549bebc884fad838e5aef43df0159)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.331788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89288b8b-9fcc-47d5-805d-43ae5ef3c127 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.331905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.339186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4f97-a1c4-4622-8b5d-7dd0fcb66180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.340809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4f97-a1c4-4622-8b5d-7dd0fcb66180 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.341241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4f97-a1c4-4622-8b5d-7dd0fcb66180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.341699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4f97-a1c4-4622-8b5d-7dd0fcb66180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.342557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b4f97-a1c4-4622-8b5d-7dd0fcb66180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.344266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b4f97-a1c4-4622-8b5d-7dd0fcb66180 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.359533 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 984/1948] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:50.365229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4f97-a1c4-4622-8b5d-7dd0fcb66180 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.377814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4f97-a1c4-4622-8b5d-7dd0fcb66180 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.383701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b5466-50eb-44fa-b7a0-24e01b9150fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.384504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b5466-50eb-44fa-b7a0-24e01b9150fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.384885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b5466-50eb-44fa-b7a0-24e01b9150fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.385259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b5466-50eb-44fa-b7a0-24e01b9150fc admin admin] PATH_INFO: `/v3/projects/e8c1e8d29d024199a9ecbbbe66b7805f/users/493f074de09b43f2b0aa2a16519d5624/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.402585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '484549bebc884fad838e5aef43df0159'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.402957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b5466-50eb-44fa-b7a0-24e01b9150fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c1e8d29d024199a9ecbbbe66b7805f, user_id=493f074de09b43f2b0aa2a16519d5624, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.406144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b5466-50eb-44fa-b7a0-24e01b9150fc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.412660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '484549bebc884fad838e5aef43df0159'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.413920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '484549bebc884fad838e5aef43df0159'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.414822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '484549bebc884fad838e5aef43df0159'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.425538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '484549bebc884fad838e5aef43df0159'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:50.433493 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 985/1949] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:50 2022] PUT /identity/v3/projects/e8c1e8d29d024199a9ecbbbe66b7805f/users/493f074de09b43f2b0aa2a16519d5624/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:50.449232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2c8ad-ea47-4ba3-bd87-4244f864a303 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.452142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2c8ad-ea47-4ba3-bd87-4244f864a303 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.453346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2c8ad-ea47-4ba3-bd87-4244f864a303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.461809 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 965/1950] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:50 2022] DELETE /identity/v3/users/484549bebc884fad838e5aef43df0159 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:50.489155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-60a2c8ad-ea47-4ba3-bd87-4244f864a303 None None] MFA Rules not processed for user `493f074de09b43f2b0aa2a16519d5624`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:50.495446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-60a2c8ad-ea47-4ba3-bd87-4244f864a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:50.559128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:50.579874 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 986/1951] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:50.584434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.586231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:50.603047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:50.673232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f89ab-905a-4940-9a21-7b8e3d054be4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.678995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.679597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.679872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.680140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.680854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.682929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.720469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2c8ad-ea47-4ba3-bd87-4244f864a303 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.727689 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 966/1952] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:50 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:50.735643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2c8ad-ea47-4ba3-bd87-4244f864a303 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.829114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85188eea-caab-4401-9ffd-65522b6b5c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.829886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.830258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85188eea-caab-4401-9ffd-65522b6b5c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.830736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85188eea-caab-4401-9ffd-65522b6b5c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.831159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85188eea-caab-4401-9ffd-65522b6b5c6c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.832862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85188eea-caab-4401-9ffd-65522b6b5c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.834730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:50.835866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85188eea-caab-4401-9ffd-65522b6b5c6c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.861513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbb3bc-91b9-4977-a04e-8a789155c327 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.869019 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 987/1953] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:50 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:50.875415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85188eea-caab-4401-9ffd-65522b6b5c6c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.882696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.883598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.884097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.884467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] PATH_INFO: `/v3/projects/67d09a67d8a941478d9b545177a141d9` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.885917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d09a67d8a941478d9b545177a141d9)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.888917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.910095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85188eea-caab-4401-9ffd-65522b6b5c6c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.923834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4669d02-0f43-4e11-aa41-e7bfe883c7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:50.924225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4669d02-0f43-4e11-aa41-e7bfe883c7de admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:50.924517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4669d02-0f43-4e11-aa41-e7bfe883c7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:50.924798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4669d02-0f43-4e11-aa41-e7bfe883c7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:50.925466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4669d02-0f43-4e11-aa41-e7bfe883c7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:50.934282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4669d02-0f43-4e11-aa41-e7bfe883c7de admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:50.962451 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 988/1954] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:50.967362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4669d02-0f43-4e11-aa41-e7bfe883c7de admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:50.985432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4669d02-0f43-4e11-aa41-e7bfe883c7de admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:50.993392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d09a67d8a941478d9b545177a141d9', 'request_id': 'req-b154bb09-7067-4405-b260-1eb6384587c3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.014962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d09a67d8a941478d9b545177a141d9', 'request_id': 'req-b154bb09-7067-4405-b260-1eb6384587c3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.033750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d09a67d8a941478d9b545177a141d9', 'request_id': 'req-b154bb09-7067-4405-b260-1eb6384587c3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.059440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00264190-6a84-4144-b41e-7bd305787b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.061305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00264190-6a84-4144-b41e-7bd305787b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.061796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00264190-6a84-4144-b41e-7bd305787b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.062003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00264190-6a84-4144-b41e-7bd305787b8c admin admin] PATH_INFO: `/v3/projects/ddb7b7cf9d104aa59aaa47a47ca8e82c/users/38f3f88287494e6f8f020498c8efe248/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.063182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] The token cache is being invalidate because project 67d09a67d8a941478d9b545177a141d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:51.063505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.066562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.068533 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 967/1955] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:50 2022] DELETE /identity/v3/projects/67d09a67d8a941478d9b545177a141d9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:51.070513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.072359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.085980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00264190-6a84-4144-b41e-7bd305787b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb7b7cf9d104aa59aaa47a47ca8e82c, user_id=38f3f88287494e6f8f020498c8efe248, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.089200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154bb09-7067-4405-b260-1eb6384587c3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.090310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00264190-6a84-4144-b41e-7bd305787b8c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.094367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.095752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.096077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.096321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] PATH_INFO: `/v3/users/ac9340771ae345a7957274f0043d0586` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.097455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9340771ae345a7957274f0043d0586)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.100026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.106876 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 989/1956] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:50 2022] PUT /identity/v3/projects/ddb7b7cf9d104aa59aaa47a47ca8e82c/users/38f3f88287494e6f8f020498c8efe248/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:51.108156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c86bf-887a-443a-ab50-e2c7b580a30b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.108442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c86bf-887a-443a-ab50-e2c7b580a30b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.108718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c86bf-887a-443a-ab50-e2c7b580a30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.128532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-af4c86bf-887a-443a-ab50-e2c7b580a30b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:51.129849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-af4c86bf-887a-443a-ab50-e2c7b580a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.149405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9340771ae345a7957274f0043d0586'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.156306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9340771ae345a7957274f0043d0586'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.156986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9340771ae345a7957274f0043d0586'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.157602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9340771ae345a7957274f0043d0586'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.167102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9340771ae345a7957274f0043d0586'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:51.173921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:51.181232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:51.187808 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 968/1957] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:51 2022] DELETE /identity/v3/users/ac9340771ae345a7957274f0043d0586 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:51.187970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.189267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.196977 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 990/1958] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:51.198380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0894c7-5902-4fe2-baeb-8ee769f39867 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.198664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0894c7-5902-4fe2-baeb-8ee769f39867 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.199080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0894c7-5902-4fe2-baeb-8ee769f39867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.219146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fd0894c7-5902-4fe2-baeb-8ee769f39867 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:51.220239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fd0894c7-5902-4fe2-baeb-8ee769f39867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.258380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99af9d-9831-4620-ae08-de608d37174e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.264765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446ba4f-24c9-4b6b-bae3-824ef6f5c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.265481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ba4f-24c9-4b6b-bae3-824ef6f5c3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.265866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ba4f-24c9-4b6b-bae3-824ef6f5c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.266523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ba4f-24c9-4b6b-bae3-824ef6f5c3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.267250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446ba4f-24c9-4b6b-bae3-824ef6f5c3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.269239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446ba4f-24c9-4b6b-bae3-824ef6f5c3a1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.285290 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 969/1959] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:51.285290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abd4fc-8b4e-44cc-8222-6f252541e83b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.285290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abd4fc-8b4e-44cc-8222-6f252541e83b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.285290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abd4fc-8b4e-44cc-8222-6f252541e83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.289861 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 991/1960] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:51.290324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0894c7-5902-4fe2-baeb-8ee769f39867 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.311180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-09abd4fc-8b4e-44cc-8222-6f252541e83b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:51.312475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-09abd4fc-8b4e-44cc-8222-6f252541e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.323417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0894c7-5902-4fe2-baeb-8ee769f39867 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.323805 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 970/1961] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:51.324654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abd4fc-8b4e-44cc-8222-6f252541e83b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.326671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-09abd4fc-8b4e-44cc-8222-6f252541e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.341287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe0669-46b9-45e5-a8e6-cff850442b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.341879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0669-46b9-45e5-a8e6-cff850442b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.342573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0669-46b9-45e5-a8e6-cff850442b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.342856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe0669-46b9-45e5-a8e6-cff850442b36 admin admin] PATH_INFO: `/v3/projects/ddb7b7cf9d104aa59aaa47a47ca8e82c/users/38f3f88287494e6f8f020498c8efe248/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.344812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe0669-46b9-45e5-a8e6-cff850442b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb7b7cf9d104aa59aaa47a47ca8e82c, user_id=38f3f88287494e6f8f020498c8efe248, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.347808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe0669-46b9-45e5-a8e6-cff850442b36 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.370699 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 992/1962] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:51 2022] PUT /identity/v3/projects/ddb7b7cf9d104aa59aaa47a47ca8e82c/users/38f3f88287494e6f8f020498c8efe248/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:51.370699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe0669-46b9-45e5-a8e6-cff850442b36 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.406477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abd4fc-8b4e-44cc-8222-6f252541e83b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.480461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe0669-46b9-45e5-a8e6-cff850442b36 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.485156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1903b-8a53-4894-88ea-0aa0fe497c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.487173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd272b25-7e3e-4d90-9f94-83b0d47e8196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.487489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1903b-8a53-4894-88ea-0aa0fe497c29 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.487919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1903b-8a53-4894-88ea-0aa0fe497c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.488300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd272b25-7e3e-4d90-9f94-83b0d47e8196 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.488516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1903b-8a53-4894-88ea-0aa0fe497c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.488758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd272b25-7e3e-4d90-9f94-83b0d47e8196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.489122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd272b25-7e3e-4d90-9f94-83b0d47e8196 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.490678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd272b25-7e3e-4d90-9f94-83b0d47e8196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:51.491088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd272b25-7e3e-4d90-9f94-83b0d47e8196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.493070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd272b25-7e3e-4d90-9f94-83b0d47e8196 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.504194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1903b-8a53-4894-88ea-0aa0fe497c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.506907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1903b-8a53-4894-88ea-0aa0fe497c29 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.513823 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 993/1963] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:51.519762 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 971/1964] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:51.520532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b301d-fe61-4b37-a6a1-1b7fe98ecce7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.521395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1903b-8a53-4894-88ea-0aa0fe497c29 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.521554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b301d-fe61-4b37-a6a1-1b7fe98ecce7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.521888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b301d-fe61-4b37-a6a1-1b7fe98ecce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.552260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1903b-8a53-4894-88ea-0aa0fe497c29 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.582733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844851b-b56b-48b3-8e09-d19860df2f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.584384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844851b-b56b-48b3-8e09-d19860df2f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.584929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844851b-b56b-48b3-8e09-d19860df2f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.585272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844851b-b56b-48b3-8e09-d19860df2f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.586069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844851b-b56b-48b3-8e09-d19860df2f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.588626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844851b-b56b-48b3-8e09-d19860df2f53 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.605151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-622b301d-fe61-4b37-a6a1-1b7fe98ecce7 None None] MFA Rules not processed for user `38f3f88287494e6f8f020498c8efe248`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:51.607523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-622b301d-fe61-4b37-a6a1-1b7fe98ecce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.635528 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 972/1965] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:51 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:51.639478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844851b-b56b-48b3-8e09-d19860df2f53 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.669895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:51.748029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844851b-b56b-48b3-8e09-d19860df2f53 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.758439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a57ec-40da-4855-8e11-7c6d0baca010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.761065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a57ec-40da-4855-8e11-7c6d0baca010 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.761408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a57ec-40da-4855-8e11-7c6d0baca010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.761884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a57ec-40da-4855-8e11-7c6d0baca010 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.762571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a57ec-40da-4855-8e11-7c6d0baca010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.764963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a57ec-40da-4855-8e11-7c6d0baca010 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.768789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:51.779294 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 994/1966] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:51.796143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b301d-fe61-4b37-a6a1-1b7fe98ecce7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.798464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-622b301d-fe61-4b37-a6a1-1b7fe98ecce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.816480 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 973/1967] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:51 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:51.823156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a57ec-40da-4855-8e11-7c6d0baca010 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.838373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a57ec-40da-4855-8e11-7c6d0baca010 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.850061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79d587-9b24-4cdb-9c7f-5c4ee05afd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.850835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d587-9b24-4cdb-9c7f-5c4ee05afd18 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.851295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d587-9b24-4cdb-9c7f-5c4ee05afd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.851576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d587-9b24-4cdb-9c7f-5c4ee05afd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.852490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79d587-9b24-4cdb-9c7f-5c4ee05afd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.857085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79d587-9b24-4cdb-9c7f-5c4ee05afd18 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.885142 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 974/1968] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:51.886687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79d587-9b24-4cdb-9c7f-5c4ee05afd18 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.890014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b301d-fe61-4b37-a6a1-1b7fe98ecce7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.902745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79d587-9b24-4cdb-9c7f-5c4ee05afd18 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.910369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679781fc-7bdd-4cee-84a6-a07854d73eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.914218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679781fc-7bdd-4cee-84a6-a07854d73eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.914679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679781fc-7bdd-4cee-84a6-a07854d73eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.915218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679781fc-7bdd-4cee-84a6-a07854d73eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.915992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679781fc-7bdd-4cee-84a6-a07854d73eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.918090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679781fc-7bdd-4cee-84a6-a07854d73eb3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:51.931945 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 975/1969] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:51.936155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679781fc-7bdd-4cee-84a6-a07854d73eb3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:51.951554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679781fc-7bdd-4cee-84a6-a07854d73eb3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:51.957533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.958937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.959309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.959622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] PATH_INFO: `/v3/projects/7864f95a786b4b41ae1e66a2ae2acee1/users/db77ee86ba0a4836990199e57f42ff49/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.960496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cf336-bc24-4042-9539-dba6255db7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:51.961262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cf336-bc24-4042-9539-dba6255db7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:51.961659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cf336-bc24-4042-9539-dba6255db7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:51.962738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cf336-bc24-4042-9539-dba6255db7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:51.965168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a47cf336-bc24-4042-9539-dba6255db7d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:51.978497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] RBAC: Authorizing `identity:create_grant(project_id=7864f95a786b4b41ae1e66a2ae2acee1, user_id=db77ee86ba0a4836990199e57f42ff49, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:51.981328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:52.008753 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 976/1970] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:51 2022] PUT /identity/v3/projects/7864f95a786b4b41ae1e66a2ae2acee1/users/db77ee86ba0a4836990199e57f42ff49/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:52.012668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:52.021904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:52.063889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47cf336-bc24-4042-9539-dba6255db7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:52.066025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47cf336-bc24-4042-9539-dba6255db7d3 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:52.080916 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 995/1971] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:52.081735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9fc87-1517-4702-83ef-8a7408bddedd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:52.082162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9fc87-1517-4702-83ef-8a7408bddedd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:52.082738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9fc87-1517-4702-83ef-8a7408bddedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:52.120365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8bc9fc87-1517-4702-83ef-8a7408bddedd None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:52.123261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8bc9fc87-1517-4702-83ef-8a7408bddedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:52.133157 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 996/1972] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:52.136210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7de9d-e676-4629-88ad-2018328501de admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:52.142690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:52.144718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9fc87-1517-4702-83ef-8a7408bddedd None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:52.147401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8bc9fc87-1517-4702-83ef-8a7408bddedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:52.150419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:52.150419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:52.151011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] PATH_INFO: `/v3/projects/40c8bb7495454e54beaaa5ff7d612adb` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:52.153535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c8bb7495454e54beaaa5ff7d612adb)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:52.155474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:52.220846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c8bb7495454e54beaaa5ff7d612adb', 'request_id': 'req-8ac62469-b9ac-4845-9b74-d671a24408d7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.232696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c8bb7495454e54beaaa5ff7d612adb', 'request_id': 'req-8ac62469-b9ac-4845-9b74-d671a24408d7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.245146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c8bb7495454e54beaaa5ff7d612adb', 'request_id': 'req-8ac62469-b9ac-4845-9b74-d671a24408d7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.247500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:52.252878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9fc87-1517-4702-83ef-8a7408bddedd None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:52.259233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] The token cache is being invalidate because project 40c8bb7495454e54beaaa5ff7d612adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:52.260102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.261378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.266882 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 977/1973] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:52 2022] DELETE /identity/v3/projects/40c8bb7495454e54beaaa5ff7d612adb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:52.268838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:52.270443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:52.308210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:52.355019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:52.356334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:52.356850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:52.357304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:52.359766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:52.383252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62469-b9ac-4845-9b74-d671a24408d7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:52.388539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:52.389183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:52.389471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:52.390250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] PATH_INFO: `/v3/users/c2eb24edb4a3424b954ae59a7e5678a8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:52.391471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2eb24edb4a3424b954ae59a7e5678a8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:52.393481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:52.460416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2eb24edb4a3424b954ae59a7e5678a8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.469228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2eb24edb4a3424b954ae59a7e5678a8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.470680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2eb24edb4a3424b954ae59a7e5678a8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.471310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2eb24edb4a3424b954ae59a7e5678a8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.477666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:52.479231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:52.482011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d223a966-04e4-4dab-9531-8cfe82912f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2eb24edb4a3424b954ae59a7e5678a8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:52.491599 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 997/1974] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:52.503471 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 978/1975] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:52 2022] DELETE /identity/v3/users/c2eb24edb4a3424b954ae59a7e5678a8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:53.104153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:53.119168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc7a22-9a91-4df6-a9ee-a637aa7ac739 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:53.246697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:53.251035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.251339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.251623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] PATH_INFO: `/v3/projects/2d9fb779c61c472e9c2ab1758695e705` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.262778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9fb779c61c472e9c2ab1758695e705)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:53.264765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:53.375136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9fb779c61c472e9c2ab1758695e705', 'request_id': 'req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:53.390651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9fb779c61c472e9c2ab1758695e705', 'request_id': 'req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:53.416032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9fb779c61c472e9c2ab1758695e705', 'request_id': 'req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:53.430467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:53.436741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] The token cache is being invalidate because project 2d9fb779c61c472e9c2ab1758695e705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:53.437237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:53.438265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6b9fefee-5eb7-4be2-8c0f-a0ac255cde9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:53.441435 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 998/1976] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:53 2022] DELETE /identity/v3/projects/2d9fb779c61c472e9c2ab1758695e705 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:53.446571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d0fc-c856-4878-9ec9-460e03c8ef49 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.447068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d0fc-c856-4878-9ec9-460e03c8ef49 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.447736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331d0fc-c856-4878-9ec9-460e03c8ef49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.471646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4331d0fc-c856-4878-9ec9-460e03c8ef49 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:53.472950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4331d0fc-c856-4878-9ec9-460e03c8ef49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:53.546505 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 979/1977] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:53.555632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2d243-2434-4427-8245-85ff4dd046c9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.555917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2d243-2434-4427-8245-85ff4dd046c9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.556190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2d243-2434-4427-8245-85ff4dd046c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.579765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f2f2d243-2434-4427-8245-85ff4dd046c9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:53.580862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f2f2d243-2434-4427-8245-85ff4dd046c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:53.595891 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 999/1978] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:53.595982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331d0fc-c856-4878-9ec9-460e03c8ef49 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:53.597129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c0b37-e64c-449d-8a12-577137f6a428 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.597408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c0b37-e64c-449d-8a12-577137f6a428 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.597697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c0b37-e64c-449d-8a12-577137f6a428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.607020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331d0fc-c856-4878-9ec9-460e03c8ef49 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:53.612595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:53.613365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.613684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.613965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.616060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:53.616375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:53.618959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:53.621295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:53.628159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f87c0b37-e64c-449d-8a12-577137f6a428 None None] MFA Rules not processed for user `db77ee86ba0a4836990199e57f42ff49`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:53.629908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f87c0b37-e64c-449d-8a12-577137f6a428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:53.637079 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 980/1979] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:36:53.641126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:53.651371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b417080-f0b0-4178-bb89-953714a17f34 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:53.655965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520435d-3ccd-41b9-9d4d-cd2b922857bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:53.656567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520435d-3ccd-41b9-9d4d-cd2b922857bc admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.656844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520435d-3ccd-41b9-9d4d-cd2b922857bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.657115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520435d-3ccd-41b9-9d4d-cd2b922857bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.657797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520435d-3ccd-41b9-9d4d-cd2b922857bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:53.659802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520435d-3ccd-41b9-9d4d-cd2b922857bc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:53.692942 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 981/1980] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:53 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:53.697288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520435d-3ccd-41b9-9d4d-cd2b922857bc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:53.720476 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1000/1981] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:53.721777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c0b37-e64c-449d-8a12-577137f6a428 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:53.723477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f87c0b37-e64c-449d-8a12-577137f6a428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:53.745492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c0b37-e64c-449d-8a12-577137f6a428 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:53.788405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520435d-3ccd-41b9-9d4d-cd2b922857bc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:53.805958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464f93d-beab-4170-b201-357fed3636aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:53.809802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464f93d-beab-4170-b201-357fed3636aa admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.810692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464f93d-beab-4170-b201-357fed3636aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.812322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464f93d-beab-4170-b201-357fed3636aa admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.814059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464f93d-beab-4170-b201-357fed3636aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:53.816935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464f93d-beab-4170-b201-357fed3636aa admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:53.835520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800029b-d45a-4364-b398-1b20f05a7b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:53.836142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800029b-d45a-4364-b398-1b20f05a7b13 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.836416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800029b-d45a-4364-b398-1b20f05a7b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.836681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800029b-d45a-4364-b398-1b20f05a7b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.837362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800029b-d45a-4364-b398-1b20f05a7b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:53.839878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800029b-d45a-4364-b398-1b20f05a7b13 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:53.857289 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 982/1982] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:53 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:53.861293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464f93d-beab-4170-b201-357fed3636aa admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:53.873531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464f93d-beab-4170-b201-357fed3636aa admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:53.878948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fece6f-f402-46fa-a942-a12bb650a635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:53.879627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fece6f-f402-46fa-a942-a12bb650a635 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.879910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fece6f-f402-46fa-a942-a12bb650a635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.880864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fece6f-f402-46fa-a942-a12bb650a635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.881587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fece6f-f402-46fa-a942-a12bb650a635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:53.887769 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1001/1983] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:53 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:53.891946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800029b-d45a-4364-b398-1b20f05a7b13 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:53.892573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fece6f-f402-46fa-a942-a12bb650a635 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:53.910300 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 983/1984] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:53.914340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fece6f-f402-46fa-a942-a12bb650a635 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:53.925261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fece6f-f402-46fa-a942-a12bb650a635 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:53.968702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800029b-d45a-4364-b398-1b20f05a7b13 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:53.974825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102f6c4-cb8a-43d5-a76e-1a75feb6b8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:53.975597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102f6c4-cb8a-43d5-a76e-1a75feb6b8ae admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.975880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102f6c4-cb8a-43d5-a76e-1a75feb6b8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.976157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102f6c4-cb8a-43d5-a76e-1a75feb6b8ae admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.976793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102f6c4-cb8a-43d5-a76e-1a75feb6b8ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:53.978937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102f6c4-cb8a-43d5-a76e-1a75feb6b8ae admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:53.991599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc460598-431c-486c-b26a-cc7af1f7cec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:53.992293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc460598-431c-486c-b26a-cc7af1f7cec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:53.992603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc460598-431c-486c-b26a-cc7af1f7cec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:53.992902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc460598-431c-486c-b26a-cc7af1f7cec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:53.993700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc460598-431c-486c-b26a-cc7af1f7cec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:53.995443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc460598-431c-486c-b26a-cc7af1f7cec6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:53.996358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:54.002351 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1002/1985] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:53 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:54.007284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102f6c4-cb8a-43d5-a76e-1a75feb6b8ae admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.010118 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 984/1986] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:54.014612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc460598-431c-486c-b26a-cc7af1f7cec6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.022204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102f6c4-cb8a-43d5-a76e-1a75feb6b8ae admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.025805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc460598-431c-486c-b26a-cc7af1f7cec6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.027491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3838d5-59d1-4376-8fb0-50911e06b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.028223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3838d5-59d1-4376-8fb0-50911e06b39d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.028496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3838d5-59d1-4376-8fb0-50911e06b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.028794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3838d5-59d1-4376-8fb0-50911e06b39d admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.029446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3838d5-59d1-4376-8fb0-50911e06b39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.030913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a78c9-88fd-4e49-98fc-b9ec38450743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.031161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3838d5-59d1-4376-8fb0-50911e06b39d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.031627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a78c9-88fd-4e49-98fc-b9ec38450743 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.031911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a78c9-88fd-4e49-98fc-b9ec38450743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.032191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a78c9-88fd-4e49-98fc-b9ec38450743 admin admin] PATH_INFO: `/v3/projects/a631de55dc5348b595779419767ded77/users/c8a1c74ec8ee4725821aa17f2555e9eb/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.043018 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1003/1987] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:54.047501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3838d5-59d1-4376-8fb0-50911e06b39d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.050431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a78c9-88fd-4e49-98fc-b9ec38450743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a631de55dc5348b595779419767ded77, user_id=c8a1c74ec8ee4725821aa17f2555e9eb, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.053803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a78c9-88fd-4e49-98fc-b9ec38450743 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.059632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3838d5-59d1-4376-8fb0-50911e06b39d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.065683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32117246-6880-4e70-9da5-b11f788c5b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.066383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117246-6880-4e70-9da5-b11f788c5b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.066673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117246-6880-4e70-9da5-b11f788c5b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.066956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117246-6880-4e70-9da5-b11f788c5b09 admin admin] PATH_INFO: `/v3/projects/42428ee5681f40718b9a855ec5d61533/users/1ad87a658f3e41f9b0acb3c2785207b4/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.071517 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 985/1988] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:54 2022] PUT /identity/v3/projects/a631de55dc5348b595779419767ded77/users/c8a1c74ec8ee4725821aa17f2555e9eb/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:54.082495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9039478-aa90-4beb-a3cd-64479b26ba2d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.082814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9039478-aa90-4beb-a3cd-64479b26ba2d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.083153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9039478-aa90-4beb-a3cd-64479b26ba2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.091657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32117246-6880-4e70-9da5-b11f788c5b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42428ee5681f40718b9a855ec5d61533, user_id=1ad87a658f3e41f9b0acb3c2785207b4, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.094954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32117246-6880-4e70-9da5-b11f788c5b09 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.111195 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1004/1989] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:54 2022] PUT /identity/v3/projects/42428ee5681f40718b9a855ec5d61533/users/1ad87a658f3e41f9b0acb3c2785207b4/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:54.115621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32117246-6880-4e70-9da5-b11f788c5b09 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.135808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a9039478-aa90-4beb-a3cd-64479b26ba2d None None] MFA Rules not processed for user `c8a1c74ec8ee4725821aa17f2555e9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:54.137168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a9039478-aa90-4beb-a3cd-64479b26ba2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:54.152899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:54.207613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32117246-6880-4e70-9da5-b11f788c5b09 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.214391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce915ad4-9408-4b06-9855-88cd497dd93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.215021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce915ad4-9408-4b06-9855-88cd497dd93f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.215722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce915ad4-9408-4b06-9855-88cd497dd93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.216006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce915ad4-9408-4b06-9855-88cd497dd93f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.216666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce915ad4-9408-4b06-9855-88cd497dd93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.218980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce915ad4-9408-4b06-9855-88cd497dd93f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.232830 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1005/1990] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:54.237371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce915ad4-9408-4b06-9855-88cd497dd93f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.252881 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 986/1991] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:54.253551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce915ad4-9408-4b06-9855-88cd497dd93f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.258964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9039478-aa90-4beb-a3cd-64479b26ba2d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.260072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c93019-5743-475d-92c0-49ee2c1fd95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.260752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93019-5743-475d-92c0-49ee2c1fd95c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.261146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93019-5743-475d-92c0-49ee2c1fd95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.261427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c93019-5743-475d-92c0-49ee2c1fd95c admin admin] PATH_INFO: `/v3/projects/42428ee5681f40718b9a855ec5d61533/users/1ad87a658f3e41f9b0acb3c2785207b4/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.263728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c93019-5743-475d-92c0-49ee2c1fd95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=42428ee5681f40718b9a855ec5d61533, user_id=1ad87a658f3e41f9b0acb3c2785207b4, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.267415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c93019-5743-475d-92c0-49ee2c1fd95c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.275600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9039478-aa90-4beb-a3cd-64479b26ba2d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.282066 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1006/1992] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:54 2022] PUT /identity/v3/projects/42428ee5681f40718b9a855ec5d61533/users/1ad87a658f3e41f9b0acb3c2785207b4/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:54.361667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa91663-2089-4cee-82e2-af46c42566ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.362380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa91663-2089-4cee-82e2-af46c42566ea admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.362592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa91663-2089-4cee-82e2-af46c42566ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.362841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa91663-2089-4cee-82e2-af46c42566ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.363488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa91663-2089-4cee-82e2-af46c42566ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.366112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa91663-2089-4cee-82e2-af46c42566ea admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.405227 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 987/1993] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:54.407063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c93019-5743-475d-92c0-49ee2c1fd95c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.430270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c93019-5743-475d-92c0-49ee2c1fd95c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.499385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59273bf7-4c68-4367-9acb-236b5228454e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.500278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273bf7-4c68-4367-9acb-236b5228454e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.500598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273bf7-4c68-4367-9acb-236b5228454e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.500978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59273bf7-4c68-4367-9acb-236b5228454e admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.501655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59273bf7-4c68-4367-9acb-236b5228454e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.509494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59273bf7-4c68-4367-9acb-236b5228454e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.554489 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1007/1994] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:54 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:54.555286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa91663-2089-4cee-82e2-af46c42566ea admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.577611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa91663-2089-4cee-82e2-af46c42566ea admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.589998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12806d9a-de6c-4470-a517-cab69e1d9385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.590637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12806d9a-de6c-4470-a517-cab69e1d9385 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.590933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12806d9a-de6c-4470-a517-cab69e1d9385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.591191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12806d9a-de6c-4470-a517-cab69e1d9385 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.591847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12806d9a-de6c-4470-a517-cab69e1d9385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.593441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12806d9a-de6c-4470-a517-cab69e1d9385 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.628627 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 988/1995] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:36:54.634109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59273bf7-4c68-4367-9acb-236b5228454e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.666530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59273bf7-4c68-4367-9acb-236b5228454e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.683236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ab6c1-addc-4241-a96b-22213bb1fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.684065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ab6c1-addc-4241-a96b-22213bb1fa99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.684350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ab6c1-addc-4241-a96b-22213bb1fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.684750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ab6c1-addc-4241-a96b-22213bb1fa99 admin admin] PATH_INFO: `/v3/projects/a201b04a59654f93b67352c24a522b13/users/3833a2b24d6340bfb9b432bd8dfb6d9e/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.705137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ab6c1-addc-4241-a96b-22213bb1fa99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a201b04a59654f93b67352c24a522b13, user_id=3833a2b24d6340bfb9b432bd8dfb6d9e, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.707948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ab6c1-addc-4241-a96b-22213bb1fa99 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.724244 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1008/1996] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:54 2022] PUT /identity/v3/projects/a201b04a59654f93b67352c24a522b13/users/3833a2b24d6340bfb9b432bd8dfb6d9e/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:54.727456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12806d9a-de6c-4470-a517-cab69e1d9385 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.741468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12806d9a-de6c-4470-a517-cab69e1d9385 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.815880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3d8c0-bac2-4887-8eb3-4bf94c2d7c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.816578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3d8c0-bac2-4887-8eb3-4bf94c2d7c1f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.816822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3d8c0-bac2-4887-8eb3-4bf94c2d7c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.817148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3d8c0-bac2-4887-8eb3-4bf94c2d7c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.818940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3d8c0-bac2-4887-8eb3-4bf94c2d7c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.820433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3d8c0-bac2-4887-8eb3-4bf94c2d7c1f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.835065 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 989/1997] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:54.836116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ab6c1-addc-4241-a96b-22213bb1fa99 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:54.848918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ab6c1-addc-4241-a96b-22213bb1fa99 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:54.858364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:54.859400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.860547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.861171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] PATH_INFO: `/v3/projects/a201b04a59654f93b67352c24a522b13/users/3833a2b24d6340bfb9b432bd8dfb6d9e/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.865668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a201b04a59654f93b67352c24a522b13, user_id=3833a2b24d6340bfb9b432bd8dfb6d9e, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:54.869039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:54.879123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:54.889717 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1009/1998] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:36:54 2022] PUT /identity/v3/projects/a201b04a59654f93b67352c24a522b13/users/3833a2b24d6340bfb9b432bd8dfb6d9e/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:54.901657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302395e-e251-4caa-9b2e-94c87432b339 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:54.901935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302395e-e251-4caa-9b2e-94c87432b339 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:54.902220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302395e-e251-4caa-9b2e-94c87432b339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:54.960534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c302395e-e251-4caa-9b2e-94c87432b339 None None] MFA Rules not processed for user `3833a2b24d6340bfb9b432bd8dfb6d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:54.962048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c302395e-e251-4caa-9b2e-94c87432b339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:54.983637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:55.061761 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 990/1999] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:55.073852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:55.075765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:55.179432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76cfed-f58f-415c-ae66-ab10090a7a3f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:55.201799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de462d-93fa-4a90-9893-3e5b8901d0f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:55.202749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de462d-93fa-4a90-9893-3e5b8901d0f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:55.203044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de462d-93fa-4a90-9893-3e5b8901d0f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:55.203329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de462d-93fa-4a90-9893-3e5b8901d0f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:55.226978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de462d-93fa-4a90-9893-3e5b8901d0f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:55.229397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de462d-93fa-4a90-9893-3e5b8901d0f0 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:55.265966 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1010/2000] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:55.280803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232448b-1bfe-4350-a2df-df0ecdc2eda4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:55.281042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232448b-1bfe-4350-a2df-df0ecdc2eda4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:55.281284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232448b-1bfe-4350-a2df-df0ecdc2eda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:55.345578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7232448b-1bfe-4350-a2df-df0ecdc2eda4 None None] MFA Rules not processed for user `1ad87a658f3e41f9b0acb3c2785207b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:55.351075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7232448b-1bfe-4350-a2df-df0ecdc2eda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:55.511731 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 991/2001] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:55.524100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3337fe-5e63-4a35-bda3-9e88296b67fe None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:55.524433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3337fe-5e63-4a35-bda3-9e88296b67fe None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:55.524745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3337fe-5e63-4a35-bda3-9e88296b67fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:55.557350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bb3337fe-5e63-4a35-bda3-9e88296b67fe None None] MFA Rules not processed for user `db77ee86ba0a4836990199e57f42ff49`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:55.558582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bb3337fe-5e63-4a35-bda3-9e88296b67fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:55.671147 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1011/2002] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:55.691521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232448b-1bfe-4350-a2df-df0ecdc2eda4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:55.693612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7232448b-1bfe-4350-a2df-df0ecdc2eda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:55.774622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3337fe-5e63-4a35-bda3-9e88296b67fe None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:55.776421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bb3337fe-5e63-4a35-bda3-9e88296b67fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:55.867313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232448b-1bfe-4350-a2df-df0ecdc2eda4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:55.882463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3337fe-5e63-4a35-bda3-9e88296b67fe None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:55.883114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d56f8-5c5c-4aae-b5ac-a78cece33994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:55.885167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d56f8-5c5c-4aae-b5ac-a78cece33994 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:55.885455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d56f8-5c5c-4aae-b5ac-a78cece33994 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:55.885762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d56f8-5c5c-4aae-b5ac-a78cece33994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:55.893208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:55.894007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:55.894409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:55.894594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] PATH_INFO: `/v3/users/d946f45a27054a0c8c7be4ea3d0d9cfb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:55.896067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d946f45a27054a0c8c7be4ea3d0d9cfb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:55.898201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:55.915473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d56f8-5c5c-4aae-b5ac-a78cece33994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:55.917935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d56f8-5c5c-4aae-b5ac-a78cece33994 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:55.955617 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 992/2003] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:56.009242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd946f45a27054a0c8c7be4ea3d0d9cfb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.011174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:56.029187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd946f45a27054a0c8c7be4ea3d0d9cfb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.030137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd946f45a27054a0c8c7be4ea3d0d9cfb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.034470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd946f45a27054a0c8c7be4ea3d0d9cfb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.050664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd946f45a27054a0c8c7be4ea3d0d9cfb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.105693 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1012/2004] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:55 2022] DELETE /identity/v3/users/d946f45a27054a0c8c7be4ea3d0d9cfb => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:56.108172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641ee3-ce4c-4a98-ba6e-fcd86a6cfa5d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:56.108705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641ee3-ce4c-4a98-ba6e-fcd86a6cfa5d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:56.109156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641ee3-ce4c-4a98-ba6e-fcd86a6cfa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:56.147534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f3641ee3-ce4c-4a98-ba6e-fcd86a6cfa5d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:56.149078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f3641ee3-ce4c-4a98-ba6e-fcd86a6cfa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:56.167441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:56.235431 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 993/2005] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:36:56.246568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:56.250079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:56.282912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3641ee3-ce4c-4a98-ba6e-fcd86a6cfa5d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:56.284332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f3641ee3-ce4c-4a98-ba6e-fcd86a6cfa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:56.298749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3641ee3-ce4c-4a98-ba6e-fcd86a6cfa5d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:56.304075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:56.304727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:56.305016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:56.305289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] PATH_INFO: `/v3/users/c8a1c74ec8ee4725821aa17f2555e9eb` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:56.312324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a1c74ec8ee4725821aa17f2555e9eb)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:56.314365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:56.334625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc5429-4c0a-4750-b63d-c02937b5b51a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:56.338914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb774a-21ea-47ed-97b1-25f8c0b8caa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:56.340129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb774a-21ea-47ed-97b1-25f8c0b8caa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:56.340129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb774a-21ea-47ed-97b1-25f8c0b8caa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:56.340129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb774a-21ea-47ed-97b1-25f8c0b8caa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:56.362067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb774a-21ea-47ed-97b1-25f8c0b8caa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:56.363938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb774a-21ea-47ed-97b1-25f8c0b8caa8 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:56.378136 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1013/2006] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:56.399897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a1c74ec8ee4725821aa17f2555e9eb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.412356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a1c74ec8ee4725821aa17f2555e9eb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.413325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a1c74ec8ee4725821aa17f2555e9eb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.414793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a1c74ec8ee4725821aa17f2555e9eb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.428416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a1c74ec8ee4725821aa17f2555e9eb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:56.461940 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 994/2007] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:56 2022] DELETE /identity/v3/users/c8a1c74ec8ee4725821aa17f2555e9eb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:56.467807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32806-837b-4df0-a84f-dd5b77629b4d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:56.468552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32806-837b-4df0-a84f-dd5b77629b4d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:56.469163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32806-837b-4df0-a84f-dd5b77629b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:56.502175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e7c32806-837b-4df0-a84f-dd5b77629b4d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:56.503411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e7c32806-837b-4df0-a84f-dd5b77629b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:56.649490 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1014/2008] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:56.660589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:56.664085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:56.749879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90823c9a-a144-46d3-ae15-47cd0434d5aa admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:56.757954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:56.758847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:56.759445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:56.760133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:56.787157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:56.792248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:56.811044 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 995/2009] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:36:57.143174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c32806-837b-4df0-a84f-dd5b77629b4d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:57.145060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e7c32806-837b-4df0-a84f-dd5b77629b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:57.165003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c32806-837b-4df0-a84f-dd5b77629b4d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:57.174466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:57.175629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:57.178556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:57.178933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] PATH_INFO: `/v3/projects/d2e34f0da33544549e5636cd9f815a90` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:57.181005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e34f0da33544549e5636cd9f815a90)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:57.183683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:57.304674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:57.306511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:57.330824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e34f0da33544549e5636cd9f815a90', 'request_id': 'req-7cb904fc-7959-4f68-b481-768503b1c625'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.353073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e34f0da33544549e5636cd9f815a90', 'request_id': 'req-7cb904fc-7959-4f68-b481-768503b1c625'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.376043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e34f0da33544549e5636cd9f815a90', 'request_id': 'req-7cb904fc-7959-4f68-b481-768503b1c625'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.384506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:57.401432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:57.416599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] The token cache is being invalidate because project d2e34f0da33544549e5636cd9f815a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:57.417421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.417810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.420201 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1015/2010] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:57 2022] DELETE /identity/v3/projects/d2e34f0da33544549e5636cd9f815a90 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:57.424087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:57.427584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:57.468121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496019c-4c8c-40a2-82ea-01acbe1d7cc7 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:57.549749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:57.561724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:57.562769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:57.563256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:57.570861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:57.600988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb904fc-7959-4f68-b481-768503b1c625 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:57.613218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:57.619160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:57.620903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:57.621190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] PATH_INFO: `/v3/users/8607a1406a26485bb208a0fc2af331b1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:57.623344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8607a1406a26485bb208a0fc2af331b1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:57.628026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:57.722046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:57.724384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:57.740263 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 996/2011] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:36:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:36:57.751705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8607a1406a26485bb208a0fc2af331b1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.761786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8607a1406a26485bb208a0fc2af331b1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.762602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8607a1406a26485bb208a0fc2af331b1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.763857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8607a1406a26485bb208a0fc2af331b1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.776742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8607a1406a26485bb208a0fc2af331b1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:57.831042 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1016/2012] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:57 2022] DELETE /identity/v3/users/8607a1406a26485bb208a0fc2af331b1 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:58.430603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:58.446804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0c21c-f4d6-4fac-a931-9fdcee9b3fbd service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:58.573062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:58.576042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:58.742531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:58.746255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:58.746597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:58.746889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] PATH_INFO: `/v3/projects/9cf0853d9cd24f07b59d94e772675fa9` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:58.749035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf0853d9cd24f07b59d94e772675fa9)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:58.756434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:58.810737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5681870-486b-4fcd-968b-fd8e992ad429 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:58.829817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:58.830845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:58.831335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:58.831809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] PATH_INFO: `/v3/projects/a631de55dc5348b595779419767ded77` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:58.834160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a631de55dc5348b595779419767ded77)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:58.836432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:58.864830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:58.872226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:58.885067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf0853d9cd24f07b59d94e772675fa9', 'request_id': 'req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:58.900744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf0853d9cd24f07b59d94e772675fa9', 'request_id': 'req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:58.908723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf0853d9cd24f07b59d94e772675fa9', 'request_id': 'req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:58.940431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] The token cache is being invalidate because project 9cf0853d9cd24f07b59d94e772675fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:58.940912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:58.942496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f14c8f69-8266-4568-9f59-c5ef8a7b4e89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:58.945087 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 997/2013] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:58 2022] DELETE /identity/v3/projects/9cf0853d9cd24f07b59d94e772675fa9 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:58.950690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c58c6-c3eb-404a-ad62-7cb6a73ff6b5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:58.951364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c58c6-c3eb-404a-ad62-7cb6a73ff6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:58.951943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c58c6-c3eb-404a-ad62-7cb6a73ff6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:58.973251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a631de55dc5348b595779419767ded77', 'request_id': 'req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:58.985869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5f5c58c6-c3eb-404a-ad62-7cb6a73ff6b5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:58.987707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5f5c58c6-c3eb-404a-ad62-7cb6a73ff6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:58.991643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a631de55dc5348b595779419767ded77', 'request_id': 'req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.006489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a631de55dc5348b595779419767ded77', 'request_id': 'req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.033853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] The token cache is being invalidate because project a631de55dc5348b595779419767ded77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:36:59.034210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.036892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.042769 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1017/2014] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:36:58 2022] DELETE /identity/v3/projects/a631de55dc5348b595779419767ded77 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:36:59.044244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:59.046314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:59.183729 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 998/2015] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:59.193762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac1d17-54d0-47bd-ba77-056aaad11d43 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:59.194285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac1d17-54d0-47bd-ba77-056aaad11d43 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:59.194773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac1d17-54d0-47bd-ba77-056aaad11d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:59.200315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be8131-a1f4-4eb7-b4f8-81ee2576d15c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:59.218578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:59.219253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:59.219531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:59.219781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] PATH_INFO: `/v3/users/3833a2b24d6340bfb9b432bd8dfb6d9e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:59.221117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3833a2b24d6340bfb9b432bd8dfb6d9e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:59.222937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:59.236745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-91ac1d17-54d0-47bd-ba77-056aaad11d43 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:36:59.240971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-91ac1d17-54d0-47bd-ba77-056aaad11d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:59.260045 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 999/2016] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:36:59.263524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac1d17-54d0-47bd-ba77-056aaad11d43 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:59.276532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac1d17-54d0-47bd-ba77-056aaad11d43 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:59.283855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:59.285075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:59.287318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:59.287603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:59.288292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:36:59.288651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:59.294005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:59.328866 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1000/2017] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:36:59.329802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:59.346138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3833a2b24d6340bfb9b432bd8dfb6d9e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.353482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff25d9-0018-467c-835a-6fbedc8366dd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:59.363767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:59.365583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:59.366140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:59.366629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:59.366857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3833a2b24d6340bfb9b432bd8dfb6d9e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.367805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3833a2b24d6340bfb9b432bd8dfb6d9e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.368331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:59.372628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:59.385761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3833a2b24d6340bfb9b432bd8dfb6d9e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.419860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3833a2b24d6340bfb9b432bd8dfb6d9e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:36:59.449702 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1001/2018] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:36:59 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:36:59.451896 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1018/2019] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:36:59 2022] DELETE /identity/v3/users/3833a2b24d6340bfb9b432bd8dfb6d9e => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:36:59.466898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:59.469885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:36:59.630709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074b6b2-b10a-431b-8dbc-56a9e36a8a25 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:59.646043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e238a-fa9f-4201-b8b1-bf903269b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:59.647131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e238a-fa9f-4201-b8b1-bf903269b0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:59.647867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e238a-fa9f-4201-b8b1-bf903269b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:59.648366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e238a-fa9f-4201-b8b1-bf903269b0c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:59.649226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e238a-fa9f-4201-b8b1-bf903269b0c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:59.651521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e238a-fa9f-4201-b8b1-bf903269b0c4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:59.730588 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1002/2020] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:36:59 2022] POST /identity/v3/users => generated 294 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:36:59.734340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:36:59.794480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:36:59.936050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310851fd-be6a-430d-98b5-c178e280bffd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:36:59.961789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:36:59.962852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:36:59.963704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:36:59.964132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:36:59.964982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:36:59.967092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:36:59.993046 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1019/2021] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:36:59.998615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e238a-fa9f-4201-b8b1-bf903269b0c4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.021918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e238a-fa9f-4201-b8b1-bf903269b0c4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.031187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651d78d-54a5-4105-bf53-06c27575efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.055255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d78d-54a5-4105-bf53-06c27575efad admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.056709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d78d-54a5-4105-bf53-06c27575efad admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.058263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d78d-54a5-4105-bf53-06c27575efad admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.059629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651d78d-54a5-4105-bf53-06c27575efad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.065778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651d78d-54a5-4105-bf53-06c27575efad admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.107718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.115142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.116323 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1003/2022] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:00.118741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651d78d-54a5-4105-bf53-06c27575efad admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.148179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bbc8d-f8f7-486c-874d-33b379034da8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.153862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651d78d-54a5-4105-bf53-06c27575efad admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.168964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a672dd-6dba-441b-a341-12d7170b8b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.171157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a672dd-6dba-441b-a341-12d7170b8b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.172527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.173999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a672dd-6dba-441b-a341-12d7170b8b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.173999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a672dd-6dba-441b-a341-12d7170b8b69 admin admin] PATH_INFO: `/v3/projects/1bc7803ffb7f4e16b9b4e76192c23c03/users/ef6d604b95454aeea3c13f6b814f65a4/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.174707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.175279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.175704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] PATH_INFO: `/v3/projects/a201b04a59654f93b67352c24a522b13` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.178606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a201b04a59654f93b67352c24a522b13)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.180855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.189508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:00.226470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a672dd-6dba-441b-a341-12d7170b8b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc7803ffb7f4e16b9b4e76192c23c03, user_id=ef6d604b95454aeea3c13f6b814f65a4, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.233758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a672dd-6dba-441b-a341-12d7170b8b69 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.271178 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1004/2023] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:00 2022] PUT /identity/v3/projects/1bc7803ffb7f4e16b9b4e76192c23c03/users/ef6d604b95454aeea3c13f6b814f65a4/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:00.282432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66fd1-d649-42d8-913f-6ae4f776c03c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.282863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66fd1-d649-42d8-913f-6ae4f776c03c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.283143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f66fd1-d649-42d8-913f-6ae4f776c03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.309409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a201b04a59654f93b67352c24a522b13', 'request_id': 'req-14947073-a7e0-4852-9830-d5aad7f7d66f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.324972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a201b04a59654f93b67352c24a522b13', 'request_id': 'req-14947073-a7e0-4852-9830-d5aad7f7d66f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.344413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a201b04a59654f93b67352c24a522b13', 'request_id': 'req-14947073-a7e0-4852-9830-d5aad7f7d66f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.359227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-37f66fd1-d649-42d8-913f-6ae4f776c03c None None] MFA Rules not processed for user `ef6d604b95454aeea3c13f6b814f65a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:00.363081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-37f66fd1-d649-42d8-913f-6ae4f776c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.372539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] The token cache is being invalidate because project a201b04a59654f93b67352c24a522b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:00.372829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.375507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14947073-a7e0-4852-9830-d5aad7f7d66f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.383824 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1020/2024] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:00 2022] DELETE /identity/v3/projects/a201b04a59654f93b67352c24a522b13 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:00.385564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b02b-b1a3-4112-bc86-4c2fa3f9a9c7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.386034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b02b-b1a3-4112-bc86-4c2fa3f9a9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.386379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11b02b-b1a3-4112-bc86-4c2fa3f9a9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.467426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3c11b02b-b1a3-4112-bc86-4c2fa3f9a9c7 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:00.473255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3c11b02b-b1a3-4112-bc86-4c2fa3f9a9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.477659 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1005/2025] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:00.486996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f66fd1-d649-42d8-913f-6ae4f776c03c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.488437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-37f66fd1-d649-42d8-913f-6ae4f776c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.612808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f66fd1-d649-42d8-913f-6ae4f776c03c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.618042 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1021/2026] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:00.624114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5eae-f7ea-4a3a-a73d-374624ad1dbe None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.624469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5eae-f7ea-4a3a-a73d-374624ad1dbe None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.624806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5eae-f7ea-4a3a-a73d-374624ad1dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.626265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.626895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.627176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.627449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.628088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.630235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.657274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ea0e5eae-f7ea-4a3a-a73d-374624ad1dbe None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:00.658762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ea0e5eae-f7ea-4a3a-a73d-374624ad1dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.662934 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1006/2027] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:00 2022] POST /identity/v3/projects => generated 389 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:00.663833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.666219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.699832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:00.734871 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1022/2028] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:00.735839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e5eae-f7ea-4a3a-a73d-374624ad1dbe None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.737478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432439-66d5-4d34-a76b-f4239641a095 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.744580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.745360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.745784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.746156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] PATH_INFO: `/v3/users/248a6a25184e4429b93b247d216f4fe7` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.747500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=248a6a25184e4429b93b247d216f4fe7)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.748409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e5eae-f7ea-4a3a-a73d-374624ad1dbe None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.749594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.751492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:00.754025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9337d-eed5-40a2-aa02-4951aaeb61d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.755157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9337d-eed5-40a2-aa02-4951aaeb61d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.755405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9337d-eed5-40a2-aa02-4951aaeb61d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.755855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9337d-eed5-40a2-aa02-4951aaeb61d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.756488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9337d-eed5-40a2-aa02-4951aaeb61d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.758437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9337d-eed5-40a2-aa02-4951aaeb61d9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.787892 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1023/2029] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:00 2022] POST /identity/v3/users => generated 301 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:00.787892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9337d-eed5-40a2-aa02-4951aaeb61d9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.797151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9337d-eed5-40a2-aa02-4951aaeb61d9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.802109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.802842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.803203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.803552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.804288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:00.804670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.806336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.809602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '248a6a25184e4429b93b247d216f4fe7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.816701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '248a6a25184e4429b93b247d216f4fe7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.817549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '248a6a25184e4429b93b247d216f4fe7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.818281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '248a6a25184e4429b93b247d216f4fe7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.819234 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1024/2030] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:00.823477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.825295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.827359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '248a6a25184e4429b93b247d216f4fe7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:00.839595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727fb5c-fcd6-41e2-b8d9-48088ea47d0e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.844950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75809512-4120-4d6e-9ea9-754ee36cecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.845553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809512-4120-4d6e-9ea9-754ee36cecce admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.846283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809512-4120-4d6e-9ea9-754ee36cecce admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.846283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75809512-4120-4d6e-9ea9-754ee36cecce admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.846749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75809512-4120-4d6e-9ea9-754ee36cecce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.851774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75809512-4120-4d6e-9ea9-754ee36cecce admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.851932 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1007/2031] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:00 2022] DELETE /identity/v3/users/248a6a25184e4429b93b247d216f4fe7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:00.851932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.853674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.860264 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1025/2032] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:00.861672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f52077-c376-4fdf-b03d-5afb6c22f34a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.861964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f52077-c376-4fdf-b03d-5afb6c22f34a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.862234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f52077-c376-4fdf-b03d-5afb6c22f34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.881248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c8f52077-c376-4fdf-b03d-5afb6c22f34a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:00.882502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c8f52077-c376-4fdf-b03d-5afb6c22f34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:00.944357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e3f52-a443-4a80-85e0-94b799ba7444 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.950953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.951722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.952004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.952274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:00.953156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:00.955188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:00.964857 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1026/2033] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:00.966069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f52077-c376-4fdf-b03d-5afb6c22f34a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:00.980215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f52077-c376-4fdf-b03d-5afb6c22f34a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:00.991274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f24dd-9e94-40e6-ba57-e813cc18dd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:00.991274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f24dd-9e94-40e6-ba57-e813cc18dd08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:00.991274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f24dd-9e94-40e6-ba57-e813cc18dd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:00.991274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f24dd-9e94-40e6-ba57-e813cc18dd08 admin admin] PATH_INFO: `/v3/projects/aa780c39b62d4642b0bd5597ba021294/users/9f595548438e48e8ab11ba54f8351716/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.003606 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1008/2034] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:00 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:01.007938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.009294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f24dd-9e94-40e6-ba57-e813cc18dd08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa780c39b62d4642b0bd5597ba021294, user_id=9f595548438e48e8ab11ba54f8351716, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.009412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:01.012481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f24dd-9e94-40e6-ba57-e813cc18dd08 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.035059 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1027/2035] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:00 2022] PUT /identity/v3/projects/aa780c39b62d4642b0bd5597ba021294/users/9f595548438e48e8ab11ba54f8351716/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:01.037781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f24dd-9e94-40e6-ba57-e813cc18dd08 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.085482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4e338-b690-4fef-84e8-fc3187509302 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.103718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:01.133169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f24dd-9e94-40e6-ba57-e813cc18dd08 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.143165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17137d30-71ed-4a5f-9296-4926d7131158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.143863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137d30-71ed-4a5f-9296-4926d7131158 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.144151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137d30-71ed-4a5f-9296-4926d7131158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.144376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3207f-26b3-48b3-86e1-a0ab0fd49b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.144494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17137d30-71ed-4a5f-9296-4926d7131158 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.145052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17137d30-71ed-4a5f-9296-4926d7131158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.145159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3207f-26b3-48b3-86e1-a0ab0fd49b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.145513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3207f-26b3-48b3-86e1-a0ab0fd49b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.146132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3207f-26b3-48b3-86e1-a0ab0fd49b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.148094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17137d30-71ed-4a5f-9296-4926d7131158 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.157086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3207f-26b3-48b3-86e1-a0ab0fd49b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.158420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3207f-26b3-48b3-86e1-a0ab0fd49b73 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.178469 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1009/2036] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:01.179283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3207f-26b3-48b3-86e1-a0ab0fd49b73 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.183013 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1028/2037] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:01 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:01.187265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17137d30-71ed-4a5f-9296-4926d7131158 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.194575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:01.198990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3207f-26b3-48b3-86e1-a0ab0fd49b73 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.201755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17137d30-71ed-4a5f-9296-4926d7131158 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.207939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c5fd-c190-4437-9427-a8e885204e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.209059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c5fd-c190-4437-9427-a8e885204e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.209259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516551a7-1d3f-409e-a7e7-7dad94d15a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.209754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c5fd-c190-4437-9427-a8e885204e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.210186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c5fd-c190-4437-9427-a8e885204e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.210661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516551a7-1d3f-409e-a7e7-7dad94d15a19 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.211374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516551a7-1d3f-409e-a7e7-7dad94d15a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.211374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516551a7-1d3f-409e-a7e7-7dad94d15a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.211870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516551a7-1d3f-409e-a7e7-7dad94d15a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.212295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c5fd-c190-4437-9427-a8e885204e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.214165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516551a7-1d3f-409e-a7e7-7dad94d15a19 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.214597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c5fd-c190-4437-9427-a8e885204e98 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.234276 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1010/2038] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:01.236078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c5fd-c190-4437-9427-a8e885204e98 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.242950 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1029/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:01.244678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516551a7-1d3f-409e-a7e7-7dad94d15a19 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.260405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c5fd-c190-4437-9427-a8e885204e98 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.268234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7099385-0617-4b29-8ed2-bd355d5049a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.269042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7099385-0617-4b29-8ed2-bd355d5049a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.269517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7099385-0617-4b29-8ed2-bd355d5049a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.269822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7099385-0617-4b29-8ed2-bd355d5049a5 admin admin] PATH_INFO: `/v3/projects/aa780c39b62d4642b0bd5597ba021294/users/9f595548438e48e8ab11ba54f8351716/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.273981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7099385-0617-4b29-8ed2-bd355d5049a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa780c39b62d4642b0bd5597ba021294, user_id=9f595548438e48e8ab11ba54f8351716, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.276748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7099385-0617-4b29-8ed2-bd355d5049a5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.289134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516551a7-1d3f-409e-a7e7-7dad94d15a19 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.293456 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1011/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:01 2022] PUT /identity/v3/projects/aa780c39b62d4642b0bd5597ba021294/users/9f595548438e48e8ab11ba54f8351716/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:01.304152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec762-bf72-431f-b985-16f2dd5df37f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.304440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec762-bf72-431f-b985-16f2dd5df37f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.304713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ec762-bf72-431f-b985-16f2dd5df37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.382748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-688ec762-bf72-431f-b985-16f2dd5df37f None None] MFA Rules not processed for user `9f595548438e48e8ab11ba54f8351716`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:01.384399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-688ec762-bf72-431f-b985-16f2dd5df37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:01.424257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d3355-bda3-4be9-a5b5-d66fb5dd6ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.425111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3355-bda3-4be9-a5b5-d66fb5dd6ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.425336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3355-bda3-4be9-a5b5-d66fb5dd6ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.425598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3355-bda3-4be9-a5b5-d66fb5dd6ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.426540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d3355-bda3-4be9-a5b5-d66fb5dd6ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.433718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d3355-bda3-4be9-a5b5-d66fb5dd6ebe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.463085 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1030/2041] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:01.469677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d3355-bda3-4be9-a5b5-d66fb5dd6ebe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.494087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d3355-bda3-4be9-a5b5-d66fb5dd6ebe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.499333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990da9b-c6e0-482b-8fd3-0f234ca30df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.500159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990da9b-c6e0-482b-8fd3-0f234ca30df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.500647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990da9b-c6e0-482b-8fd3-0f234ca30df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.500929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990da9b-c6e0-482b-8fd3-0f234ca30df9 admin admin] PATH_INFO: `/v3/projects/3a634055eea14346bd2ec2e14151633b/users/a1d3816451a8467f9afe467b8866b53d/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.533217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990da9b-c6e0-482b-8fd3-0f234ca30df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a634055eea14346bd2ec2e14151633b, user_id=a1d3816451a8467f9afe467b8866b53d, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.542992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990da9b-c6e0-482b-8fd3-0f234ca30df9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.559017 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1012/2042] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:01 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:01.571387 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1031/2043] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:01 2022] PUT /identity/v3/projects/3a634055eea14346bd2ec2e14151633b/users/a1d3816451a8467f9afe467b8866b53d/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:01.572717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ec762-bf72-431f-b985-16f2dd5df37f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.575227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-688ec762-bf72-431f-b985-16f2dd5df37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:01.581992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8370a6-f385-4452-970e-041ef5ccba08 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.582295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8370a6-f385-4452-970e-041ef5ccba08 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.582541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8370a6-f385-4452-970e-041ef5ccba08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.617264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6c8370a6-f385-4452-970e-041ef5ccba08 None None] MFA Rules not processed for user `a1d3816451a8467f9afe467b8866b53d`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:01.618760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6c8370a6-f385-4452-970e-041ef5ccba08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:01.683555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ec762-bf72-431f-b985-16f2dd5df37f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.749668 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1032/2044] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:01.772632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8370a6-f385-4452-970e-041ef5ccba08 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:01.774197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6c8370a6-f385-4452-970e-041ef5ccba08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:01.777430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:01.807488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2674cc9-837b-465d-b1f7-9aa8624f43d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.809021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2674cc9-837b-465d-b1f7-9aa8624f43d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.809832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2674cc9-837b-465d-b1f7-9aa8624f43d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.810750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2674cc9-837b-465d-b1f7-9aa8624f43d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.876224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:01.931949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8370a6-f385-4452-970e-041ef5ccba08 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:01.940432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fe7a090bd7941bb827d301aa4949857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:01.941268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:01.942089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2674cc9-837b-465d-b1f7-9aa8624f43d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.942418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:01.943232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:01.944451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2674cc9-837b-465d-b1f7-9aa8624f43d5 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.954515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:01.956669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:01.957588 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1013/2045] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:01 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:01.978694 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1033/2046] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:02.220619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2674cc9-837b-465d-b1f7-9aa8624f43d5 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:02.356413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2674cc9-837b-465d-b1f7-9aa8624f43d5 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:02.364878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:02.366029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:02.366928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:02.367313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] PATH_INFO: `/v3/users/ef6d604b95454aeea3c13f6b814f65a4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:02.368824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6d604b95454aeea3c13f6b814f65a4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:02.371538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:02.487861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:02.490104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:02.493759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6d604b95454aeea3c13f6b814f65a4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.513280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6d604b95454aeea3c13f6b814f65a4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.514943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8bbb4-6f41-4a99-9e26-601bb3719fb5 service glance] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:02.515761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6d604b95454aeea3c13f6b814f65a4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.516395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6d604b95454aeea3c13f6b814f65a4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.529991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:02.530681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:02.530957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:02.531232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] PATH_INFO: `/v3/projects/224e853b2f0740cc838a60ae46d63a2f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:02.535220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=224e853b2f0740cc838a60ae46d63a2f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:02.538522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:02.548589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6fb711c0-685d-4c11-a856-7ae6cf93e9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6d604b95454aeea3c13f6b814f65a4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.580278 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1014/2047] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:02 2022] DELETE /identity/v3/users/ef6d604b95454aeea3c13f6b814f65a4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:02.587551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd14adc-8f99-407b-ae15-1a2aa7079845 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:02.587868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd14adc-8f99-407b-ae15-1a2aa7079845 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:02.588208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd14adc-8f99-407b-ae15-1a2aa7079845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:02.616379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5cd14adc-8f99-407b-ae15-1a2aa7079845 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:02.618035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5cd14adc-8f99-407b-ae15-1a2aa7079845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:02.638681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:02.655535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224e853b2f0740cc838a60ae46d63a2f', 'request_id': 'req-dc66d0f4-0534-4875-b133-1338e2087eed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.670088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224e853b2f0740cc838a60ae46d63a2f', 'request_id': 'req-dc66d0f4-0534-4875-b133-1338e2087eed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.678199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224e853b2f0740cc838a60ae46d63a2f', 'request_id': 'req-dc66d0f4-0534-4875-b133-1338e2087eed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.722377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] The token cache is being invalidate because project 224e853b2f0740cc838a60ae46d63a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:02.722377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.722377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:02.726819 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1034/2048] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:02 2022] DELETE /identity/v3/projects/224e853b2f0740cc838a60ae46d63a2f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:02.731852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:02.737311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:02.792958 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1015/2049] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:02.810918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd14adc-8f99-407b-ae15-1a2aa7079845 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:02.812697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5cd14adc-8f99-407b-ae15-1a2aa7079845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:02.886137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d0f4-0534-4875-b133-1338e2087eed admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:02.899058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:02.900128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:02.900625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:02.901045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] PATH_INFO: `/v3/users/a9a46dd480324b2f997c43dbbc67ddaa` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:02.902655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a46dd480324b2f997c43dbbc67ddaa)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:02.905241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:02.950655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd14adc-8f99-407b-ae15-1a2aa7079845 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:03.006535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a46dd480324b2f997c43dbbc67ddaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.021336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612de1ff-ba49-4ed7-862a-7607ba9847c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:03.023185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612de1ff-ba49-4ed7-862a-7607ba9847c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:03.023483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612de1ff-ba49-4ed7-862a-7607ba9847c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:03.023761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612de1ff-ba49-4ed7-862a-7607ba9847c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:03.026931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a46dd480324b2f997c43dbbc67ddaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.027673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a46dd480324b2f997c43dbbc67ddaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.028498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a46dd480324b2f997c43dbbc67ddaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.039582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a46dd480324b2f997c43dbbc67ddaa'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.042034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:03.068750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612de1ff-ba49-4ed7-862a-7607ba9847c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:03.070765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612de1ff-ba49-4ed7-862a-7607ba9847c5 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:03.102649 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1035/2050] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:02 2022] DELETE /identity/v3/users/a9a46dd480324b2f997c43dbbc67ddaa => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:03.118032 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1016/2051] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:03.371409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:03.373361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:03.491871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100acf-7214-4ca1-95f3-57fdcaacfbae admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:03.502016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:03.503143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:03.503427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:03.503782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] PATH_INFO: `/v3/users/493f074de09b43f2b0aa2a16519d5624` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:03.506495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=493f074de09b43f2b0aa2a16519d5624)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:03.508510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:03.568798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493f074de09b43f2b0aa2a16519d5624'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.588538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493f074de09b43f2b0aa2a16519d5624'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.589370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493f074de09b43f2b0aa2a16519d5624'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.590069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493f074de09b43f2b0aa2a16519d5624'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.600457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493f074de09b43f2b0aa2a16519d5624'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.629347 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1036/2052] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:03 2022] DELETE /identity/v3/users/493f074de09b43f2b0aa2a16519d5624 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:03.631991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34205234-5048-4eed-a4cf-3604e218a3c9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:03.633305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34205234-5048-4eed-a4cf-3604e218a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:03.633990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34205234-5048-4eed-a4cf-3604e218a3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:03.661501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-34205234-5048-4eed-a4cf-3604e218a3c9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:03.663282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-34205234-5048-4eed-a4cf-3604e218a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:03.785052 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1017/2053] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:03.791534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:03.794061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:03.804025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34205234-5048-4eed-a4cf-3604e218a3c9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:03.807383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-34205234-5048-4eed-a4cf-3604e218a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:03.816800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e48a6-f63e-4201-9e59-e55e07dee579 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:03.832603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:03.833442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:03.833909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:03.834210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] PATH_INFO: `/v3/projects/1bc7803ffb7f4e16b9b4e76192c23c03` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:03.835365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc7803ffb7f4e16b9b4e76192c23c03)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:03.838119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:03.848282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:03.935099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc7803ffb7f4e16b9b4e76192c23c03', 'request_id': 'req-34cb0815-151c-45a5-90de-fc7c1740c97e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.939516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:03.953027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc7803ffb7f4e16b9b4e76192c23c03', 'request_id': 'req-34cb0815-151c-45a5-90de-fc7c1740c97e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.955026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34205234-5048-4eed-a4cf-3604e218a3c9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:03.964377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc7803ffb7f4e16b9b4e76192c23c03', 'request_id': 'req-34cb0815-151c-45a5-90de-fc7c1740c97e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.991135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] The token cache is being invalidate because project 1bc7803ffb7f4e16b9b4e76192c23c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:03.991610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.992605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:03.995293 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1037/2054] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:03 2022] DELETE /identity/v3/projects/1bc7803ffb7f4e16b9b4e76192c23c03 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:03.999968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:04.003211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:04.023054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:04.023862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.024212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.025767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.030427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:04.118565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb0815-151c-45a5-90de-fc7c1740c97e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:04.149709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:04.150922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.151361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.151787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] PATH_INFO: `/v3/users/9f595548438e48e8ab11ba54f8351716` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.153978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f595548438e48e8ab11ba54f8351716)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:04.155410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:04.156866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:04.157041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:04.182643 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1018/2055] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:04.183611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:04.187959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:04.202672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052bbf-6efb-4492-9297-ec30bb5fa92b service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:04.214680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:04.215361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.215716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.216320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] PATH_INFO: `/v3/projects/5945ebed866f429792d0c60118b19c3c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.217942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5945ebed866f429792d0c60118b19c3c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:04.221485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:04.245954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f595548438e48e8ab11ba54f8351716'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.264983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f595548438e48e8ab11ba54f8351716'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.266441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f595548438e48e8ab11ba54f8351716'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.267514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f595548438e48e8ab11ba54f8351716'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.281963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbb34fe0-3340-43e4-9e23-bc9ca4cac2a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f595548438e48e8ab11ba54f8351716'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.334978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5945ebed866f429792d0c60118b19c3c', 'request_id': 'req-16b49519-4ee5-4466-baf8-ceb80a36f86a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.359698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5945ebed866f429792d0c60118b19c3c', 'request_id': 'req-16b49519-4ee5-4466-baf8-ceb80a36f86a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.362538 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1038/2056] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:03 2022] DELETE /identity/v3/users/9f595548438e48e8ab11ba54f8351716 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:04.374144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5945ebed866f429792d0c60118b19c3c', 'request_id': 'req-16b49519-4ee5-4466-baf8-ceb80a36f86a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.382227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:04.388115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] The token cache is being invalidate because project 5945ebed866f429792d0c60118b19c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:04.388891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.390004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-16b49519-4ee5-4466-baf8-ceb80a36f86a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:04.398504 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1019/2057] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:04 2022] DELETE /identity/v3/projects/5945ebed866f429792d0c60118b19c3c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:04.398748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14feae4d-d0f4-43a9-84f9-4cdd2a8a454e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.398748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14feae4d-d0f4-43a9-84f9-4cdd2a8a454e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.398748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14feae4d-d0f4-43a9-84f9-4cdd2a8a454e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.431635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-14feae4d-d0f4-43a9-84f9-4cdd2a8a454e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:04.432851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-14feae4d-d0f4-43a9-84f9-4cdd2a8a454e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:04.536917 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1039/2058] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:04.545807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b884fd-e7cd-4e9a-afb7-a351b7b73468 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.545807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b884fd-e7cd-4e9a-afb7-a351b7b73468 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.546092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b884fd-e7cd-4e9a-afb7-a351b7b73468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.573220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-56b884fd-e7cd-4e9a-afb7-a351b7b73468 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:04.575179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-56b884fd-e7cd-4e9a-afb7-a351b7b73468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:04.587821 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1020/2059] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:04.589056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14feae4d-d0f4-43a9-84f9-4cdd2a8a454e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:04.595445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:04.635229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14feae4d-d0f4-43a9-84f9-4cdd2a8a454e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:04.641094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:04.641819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.642101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.642381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.643145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:04.643519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:04.646668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:04.669363 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1040/2060] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:04.670617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b884fd-e7cd-4e9a-afb7-a351b7b73468 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:04.687493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b884fd-e7cd-4e9a-afb7-a351b7b73468 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:04.693211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4710fc3-f325-4ba1-ba9a-dad4a6a6a176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:04.694155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4710fc3-f325-4ba1-ba9a-dad4a6a6a176 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.694332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4710fc3-f325-4ba1-ba9a-dad4a6a6a176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.694619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4710fc3-f325-4ba1-ba9a-dad4a6a6a176 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.695256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4710fc3-f325-4ba1-ba9a-dad4a6a6a176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:04.697208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4710fc3-f325-4ba1-ba9a-dad4a6a6a176 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:04.749160 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1021/2061] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:04 2022] POST /identity/v3/projects => generated 369 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:04.753054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:04.769435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f9f2-48e8-47a5-a86b-2ff95336e932 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:04.904091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:04.904811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.905095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.905373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.906473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:04.908639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:04.946244 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1041/2062] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:04 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:04.950635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4710fc3-f325-4ba1-ba9a-dad4a6a6a176 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:04.962830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4710fc3-f325-4ba1-ba9a-dad4a6a6a176 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:04.969821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:04.971739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:04.979861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a42ee-3289-4476-82f8-11f3a448fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:04.981618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a42ee-3289-4476-82f8-11f3a448fb61 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.981936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a42ee-3289-4476-82f8-11f3a448fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.982218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a42ee-3289-4476-82f8-11f3a448fb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.982893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a42ee-3289-4476-82f8-11f3a448fb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:04.984546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a42ee-3289-4476-82f8-11f3a448fb61 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:04.986400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f3a2c-c89b-41b2-a6c8-46ae3938d60b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:04.993120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:04.994001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:04.994731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:04.995233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] PATH_INFO: `/v3/projects/e8c1e8d29d024199a9ecbbbe66b7805f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:04.996729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c1e8d29d024199a9ecbbbe66b7805f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.004020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.013244 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1022/2063] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:05.020011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a42ee-3289-4476-82f8-11f3a448fb61 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.056725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a42ee-3289-4476-82f8-11f3a448fb61 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.064464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.065093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.065429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.068805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.070620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.072709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.087832 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1023/2064] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:05.089198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.092121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:05.101161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c1e8d29d024199a9ecbbbe66b7805f', 'request_id': 'req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.108598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656d552-13ae-4d83-8a00-683772323b37 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.111868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c1e8d29d024199a9ecbbbe66b7805f', 'request_id': 'req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.123931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c1e8d29d024199a9ecbbbe66b7805f', 'request_id': 'req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.139701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] The token cache is being invalidate because project e8c1e8d29d024199a9ecbbbe66b7805f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:05.140105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.140936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.146456 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1042/2065] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:04 2022] DELETE /identity/v3/projects/e8c1e8d29d024199a9ecbbbe66b7805f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:05.146456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.146456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:05.173397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.174163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.174492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.174824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] PATH_INFO: `/v3/projects/aa780c39b62d4642b0bd5597ba021294` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.176144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa780c39b62d4642b0bd5597ba021294)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.178217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.238549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f35e3-8e9f-48c4-aa52-3ee0350c53e7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.242272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa780c39b62d4642b0bd5597ba021294', 'request_id': 'req-1daef12e-7dac-4049-b81a-920a8957c3b9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.244199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:05.250635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa780c39b62d4642b0bd5597ba021294', 'request_id': 'req-1daef12e-7dac-4049-b81a-920a8957c3b9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.258654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa780c39b62d4642b0bd5597ba021294', 'request_id': 'req-1daef12e-7dac-4049-b81a-920a8957c3b9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.269542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] The token cache is being invalidate because project aa780c39b62d4642b0bd5597ba021294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:05.270010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.271562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.273215 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1024/2066] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:05 2022] DELETE /identity/v3/projects/aa780c39b62d4642b0bd5597ba021294 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:05.274050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.278546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:05.323818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19972371-b541-4928-b575-fa1abaead1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.324787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972371-b541-4928-b575-fa1abaead1fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.325232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972371-b541-4928-b575-fa1abaead1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.325509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972371-b541-4928-b575-fa1abaead1fe admin admin] PATH_INFO: `/v3/projects/93a2bc6de070450da7993395b38458ed/users/2fec55ae6e2e46deaaa74c1d0b20d11e/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.338000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:05.345145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19972371-b541-4928-b575-fa1abaead1fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a2bc6de070450da7993395b38458ed, user_id=2fec55ae6e2e46deaaa74c1d0b20d11e, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.349801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19972371-b541-4928-b575-fa1abaead1fe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.365814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daef12e-7dac-4049-b81a-920a8957c3b9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.376177 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1043/2067] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:05 2022] PUT /identity/v3/projects/93a2bc6de070450da7993395b38458ed/users/2fec55ae6e2e46deaaa74c1d0b20d11e/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:05.376177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba22a0-8395-4075-86b2-8609129f5795 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.376177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba22a0-8395-4075-86b2-8609129f5795 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.376177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba22a0-8395-4075-86b2-8609129f5795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.397682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e0ba22a0-8395-4075-86b2-8609129f5795 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:05.401224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e0ba22a0-8395-4075-86b2-8609129f5795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:05.437534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.438229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.438513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.438932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] PATH_INFO: `/v3/users/38f3f88287494e6f8f020498c8efe248` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.440159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f3f88287494e6f8f020498c8efe248)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.442244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.479338 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1044/2068] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:05.480443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8edce-2faa-4bb8-84f6-f2e5d5adb40e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.480731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8edce-2faa-4bb8-84f6-f2e5d5adb40e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.481010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8edce-2faa-4bb8-84f6-f2e5d5adb40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.530703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-eaa8edce-2faa-4bb8-84f6-f2e5d5adb40e None None] MFA Rules not processed for user `2fec55ae6e2e46deaaa74c1d0b20d11e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:05.531053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f3f88287494e6f8f020498c8efe248'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.532456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eaa8edce-2faa-4bb8-84f6-f2e5d5adb40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:05.540954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f3f88287494e6f8f020498c8efe248'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.544413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f3f88287494e6f8f020498c8efe248'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.544413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f3f88287494e6f8f020498c8efe248'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.556846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be9db27f-7f90-41e6-98e6-fa2f702dd5de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f3f88287494e6f8f020498c8efe248'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:05.588709 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1025/2069] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:05 2022] DELETE /identity/v3/users/38f3f88287494e6f8f020498c8efe248 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:05.590350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92dc94-0bb4-48d7-833d-67e960b7f990 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.590867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92dc94-0bb4-48d7-833d-67e960b7f990 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.591350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92dc94-0bb4-48d7-833d-67e960b7f990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.614857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8a92dc94-0bb4-48d7-833d-67e960b7f990 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:05.616136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8a92dc94-0bb4-48d7-833d-67e960b7f990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:05.665295 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1045/2070] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:05.666420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8edce-2faa-4bb8-84f6-f2e5d5adb40e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.670166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eaa8edce-2faa-4bb8-84f6-f2e5d5adb40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:05.715180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8edce-2faa-4bb8-84f6-f2e5d5adb40e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.716244 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1026/2071] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:05.721311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92dc94-0bb4-48d7-833d-67e960b7f990 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.727483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9d5e3-8f4a-45e9-8c74-8c03deeffeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.728230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9d5e3-8f4a-45e9-8c74-8c03deeffeb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.728614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9d5e3-8f4a-45e9-8c74-8c03deeffeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.729004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9d5e3-8f4a-45e9-8c74-8c03deeffeb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.729754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9d5e3-8f4a-45e9-8c74-8c03deeffeb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.729944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:05.735190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9d5e3-8f4a-45e9-8c74-8c03deeffeb7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.735268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92dc94-0bb4-48d7-833d-67e960b7f990 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.751287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.751919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.752351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.752641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.753336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:05.753600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.756397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.762894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:05.778614 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1046/2072] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:05 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:05.781938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9d5e3-8f4a-45e9-8c74-8c03deeffeb7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.794110 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1027/2073] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:05.799082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.829804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5761e5e-6a32-4637-9913-95a8c29e48ed admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.874597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9d5e3-8f4a-45e9-8c74-8c03deeffeb7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.883152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882a3b6-4994-49b4-a66c-7ffd2eba5597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.883722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882a3b6-4994-49b4-a66c-7ffd2eba5597 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.889870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882a3b6-4994-49b4-a66c-7ffd2eba5597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.890247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882a3b6-4994-49b4-a66c-7ffd2eba5597 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.891244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882a3b6-4994-49b4-a66c-7ffd2eba5597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.894147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882a3b6-4994-49b4-a66c-7ffd2eba5597 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.912279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8d263-7f45-440a-b983-1a0aa04d76e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.912947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8d263-7f45-440a-b983-1a0aa04d76e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.913372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8d263-7f45-440a-b983-1a0aa04d76e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.913936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8d263-7f45-440a-b983-1a0aa04d76e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.914969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8d263-7f45-440a-b983-1a0aa04d76e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.918747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8d263-7f45-440a-b983-1a0aa04d76e2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:05.933930 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1047/2074] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:05 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:05.935391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882a3b6-4994-49b4-a66c-7ffd2eba5597 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:05.949013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882a3b6-4994-49b4-a66c-7ffd2eba5597 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:05.970326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29a3e5-4a7d-4046-aae5-efb8e03c331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:05.974081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a3e5-4a7d-4046-aae5-efb8e03c331f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:05.974640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a3e5-4a7d-4046-aae5-efb8e03c331f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:05.979910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a3e5-4a7d-4046-aae5-efb8e03c331f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:05.980741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29a3e5-4a7d-4046-aae5-efb8e03c331f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:05.993567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29a3e5-4a7d-4046-aae5-efb8e03c331f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.001178 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1028/2075] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:05 2022] POST /identity/v3/projects => generated 407 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:06.002352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8d263-7f45-440a-b983-1a0aa04d76e2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.015204 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1048/2076] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:06.019274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29a3e5-4a7d-4046-aae5-efb8e03c331f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.031976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29a3e5-4a7d-4046-aae5-efb8e03c331f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.092350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8d263-7f45-440a-b983-1a0aa04d76e2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.098181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.099553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.099928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.100264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.100908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.103159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.109150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da0e9-6048-4d60-8dbe-00f2cbaae052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.109957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da0e9-6048-4d60-8dbe-00f2cbaae052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.110239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da0e9-6048-4d60-8dbe-00f2cbaae052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.110586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da0e9-6048-4d60-8dbe-00f2cbaae052 admin admin] PATH_INFO: `/v3/projects/eae8bcadd2b0420ea88bd8ae51c3f01b/users/566d9b1b65994649b969cf96be8de89e/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.130346 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1029/2077] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/users => generated 304 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:06.133799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734da0e9-6048-4d60-8dbe-00f2cbaae052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae8bcadd2b0420ea88bd8ae51c3f01b, user_id=566d9b1b65994649b969cf96be8de89e, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.134124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.135848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:06.136531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734da0e9-6048-4d60-8dbe-00f2cbaae052 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.150376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef02a617-296c-45f8-b757-e98c602a7cfb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.150554 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1049/2078] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:06 2022] PUT /identity/v3/projects/eae8bcadd2b0420ea88bd8ae51c3f01b/users/566d9b1b65994649b969cf96be8de89e/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:06.151401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da0e9-6048-4d60-8dbe-00f2cbaae052 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.219189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.219825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.220106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.220388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] PATH_INFO: `/v3/projects/ddb7b7cf9d104aa59aaa47a47ca8e82c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.221256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da0e9-6048-4d60-8dbe-00f2cbaae052 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.221504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb7b7cf9d104aa59aaa47a47ca8e82c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.224409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.227188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bd096-b186-4935-a5c1-1bad43902088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.227802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bd096-b186-4935-a5c1-1bad43902088 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.228085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bd096-b186-4935-a5c1-1bad43902088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.228357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bd096-b186-4935-a5c1-1bad43902088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.229117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bd096-b186-4935-a5c1-1bad43902088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.230787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bd096-b186-4935-a5c1-1bad43902088 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.233345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:06.234876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:06.247407 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1050/2079] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:06.247407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bd096-b186-4935-a5c1-1bad43902088 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.255947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bd096-b186-4935-a5c1-1bad43902088 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.262293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caccc4-9aed-416c-9d42-65b0184d00a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.262827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caccc4-9aed-416c-9d42-65b0184d00a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.263113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caccc4-9aed-416c-9d42-65b0184d00a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.263392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caccc4-9aed-416c-9d42-65b0184d00a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.264186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caccc4-9aed-416c-9d42-65b0184d00a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.265876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caccc4-9aed-416c-9d42-65b0184d00a5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.279789 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1051/2080] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:06.280636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caccc4-9aed-416c-9d42-65b0184d00a5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.288513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb7b7cf9d104aa59aaa47a47ca8e82c', 'request_id': 'req-52f15932-4128-4383-98bb-c71e87f64329'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:06.292473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caccc4-9aed-416c-9d42-65b0184d00a5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.296448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb7b7cf9d104aa59aaa47a47ca8e82c', 'request_id': 'req-52f15932-4128-4383-98bb-c71e87f64329'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:06.304413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb7b7cf9d104aa59aaa47a47ca8e82c', 'request_id': 'req-52f15932-4128-4383-98bb-c71e87f64329'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:06.316815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] The token cache is being invalidate because project ddb7b7cf9d104aa59aaa47a47ca8e82c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:06.317699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:06.319247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:06.320769 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1030/2081] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:06 2022] DELETE /identity/v3/projects/ddb7b7cf9d104aa59aaa47a47ca8e82c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:06.321586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.322898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:06.355287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e17f28-a951-4008-afb5-fec6e3921ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.355812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e17f28-a951-4008-afb5-fec6e3921ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.356094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e17f28-a951-4008-afb5-fec6e3921ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.356373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e17f28-a951-4008-afb5-fec6e3921ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.357021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e17f28-a951-4008-afb5-fec6e3921ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.358702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e17f28-a951-4008-afb5-fec6e3921ca8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.369269 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1052/2082] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:06.371404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeb741-0fb6-471e-952b-deaeeaa6a65a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.371696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeb741-0fb6-471e-952b-deaeeaa6a65a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.372257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeb741-0fb6-471e-952b-deaeeaa6a65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.393643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-afdeb741-0fb6-471e-952b-deaeeaa6a65a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:06.393977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f15932-4128-4383-98bb-c71e87f64329 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.394857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-afdeb741-0fb6-471e-952b-deaeeaa6a65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:06.399064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5643483-b387-4f28-8fc2-72967fb1edf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.399802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5643483-b387-4f28-8fc2-72967fb1edf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.400080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5643483-b387-4f28-8fc2-72967fb1edf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.400350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5643483-b387-4f28-8fc2-72967fb1edf2 admin admin] PATH_INFO: `/v3/projects/eae8bcadd2b0420ea88bd8ae51c3f01b/users/566d9b1b65994649b969cf96be8de89e/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.406431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5643483-b387-4f28-8fc2-72967fb1edf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae8bcadd2b0420ea88bd8ae51c3f01b, user_id=566d9b1b65994649b969cf96be8de89e, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.406431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5643483-b387-4f28-8fc2-72967fb1edf2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.408035 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1053/2083] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:06.410671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdeb741-0fb6-471e-952b-deaeeaa6a65a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.416520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-afdeb741-0fb6-471e-952b-deaeeaa6a65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:06.418457 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1031/2084] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:06 2022] PUT /identity/v3/projects/eae8bcadd2b0420ea88bd8ae51c3f01b/users/566d9b1b65994649b969cf96be8de89e/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:06.419886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ddd3-07a1-4b25-86db-1c72b70823e5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.420166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ddd3-07a1-4b25-86db-1c72b70823e5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.420434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61ddd3-07a1-4b25-86db-1c72b70823e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.430160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdeb741-0fb6-471e-952b-deaeeaa6a65a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.442328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-aa61ddd3-07a1-4b25-86db-1c72b70823e5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:06.443619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-aa61ddd3-07a1-4b25-86db-1c72b70823e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:06.490284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d23035-cb57-438a-bed1-4df83e3593d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.490975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d23035-cb57-438a-bed1-4df83e3593d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.491249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d23035-cb57-438a-bed1-4df83e3593d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.491528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d23035-cb57-438a-bed1-4df83e3593d0 admin admin] PATH_INFO: `/v3/projects/bf26bfcc5c98442ab5af3717927f7038/users/e905de725f9749319651c6eb2a1df8c0/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.505723 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1032/2085] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:06.507130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b46fa7-06b9-4027-bcf8-eb77f8e4a8b1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.507416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b46fa7-06b9-4027-bcf8-eb77f8e4a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.507686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b46fa7-06b9-4027-bcf8-eb77f8e4a8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.508005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d23035-cb57-438a-bed1-4df83e3593d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf26bfcc5c98442ab5af3717927f7038, user_id=e905de725f9749319651c6eb2a1df8c0, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.510893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d23035-cb57-438a-bed1-4df83e3593d0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.525329 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1054/2086] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:06 2022] PUT /identity/v3/projects/bf26bfcc5c98442ab5af3717927f7038/users/e905de725f9749319651c6eb2a1df8c0/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:06.526467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d23035-cb57-438a-bed1-4df83e3593d0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.538432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-18b46fa7-06b9-4027-bcf8-eb77f8e4a8b1 None None] MFA Rules not processed for user `566d9b1b65994649b969cf96be8de89e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:06.539767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-18b46fa7-06b9-4027-bcf8-eb77f8e4a8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:06.559213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:06.610192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d23035-cb57-438a-bed1-4df83e3593d0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.615438 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1033/2087] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:06.616830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aaad7-a1e4-4eb1-bd4a-da3148cf92e6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.617116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aaad7-a1e4-4eb1-bd4a-da3148cf92e6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.617386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50aaad7-a1e4-4eb1-bd4a-da3148cf92e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.618194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.618810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.619093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.619369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.620026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:06.620349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.622010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.623913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:06.643332 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1055/2088] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:06.643332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.643332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:06.655687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e50aaad7-a1e4-4eb1-bd4a-da3148cf92e6 None None] MFA Rules not processed for user `e905de725f9749319651c6eb2a1df8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:06.656913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e50aaad7-a1e4-4eb1-bd4a-da3148cf92e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:06.702978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49109e-6ea3-4b71-9737-84cd06bcd47c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.706715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb2dc-b02e-41db-a20d-359e5eead9a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.707327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb2dc-b02e-41db-a20d-359e5eead9a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.707619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb2dc-b02e-41db-a20d-359e5eead9a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.708441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb2dc-b02e-41db-a20d-359e5eead9a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.717229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcb2dc-b02e-41db-a20d-359e5eead9a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.718698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcb2dc-b02e-41db-a20d-359e5eead9a5 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.725930 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1034/2089] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:06.726753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50aaad7-a1e4-4eb1-bd4a-da3148cf92e6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.732516 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1056/2090] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:06.733354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb2dc-b02e-41db-a20d-359e5eead9a5 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.741727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50aaad7-a1e4-4eb1-bd4a-da3148cf92e6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.744157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb2dc-b02e-41db-a20d-359e5eead9a5 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.746470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110730-63d6-4cc4-b581-111b0afaf43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.747122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110730-63d6-4cc4-b581-111b0afaf43f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.747408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110730-63d6-4cc4-b581-111b0afaf43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.747813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da110730-63d6-4cc4-b581-111b0afaf43f admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.748461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da110730-63d6-4cc4-b581-111b0afaf43f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.749224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf10d-fd68-4190-a4f5-a31f6e8e437d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.749977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf10d-fd68-4190-a4f5-a31f6e8e437d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.750236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf10d-fd68-4190-a4f5-a31f6e8e437d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.750500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da110730-63d6-4cc4-b581-111b0afaf43f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.750682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf10d-fd68-4190-a4f5-a31f6e8e437d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.751115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf10d-fd68-4190-a4f5-a31f6e8e437d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.752985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf10d-fd68-4190-a4f5-a31f6e8e437d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.791064 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1057/2091] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/projects => generated 407 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:06.793074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf10d-fd68-4190-a4f5-a31f6e8e437d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.798541 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1035/2092] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:06.802879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110730-63d6-4cc4-b581-111b0afaf43f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.878152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da110730-63d6-4cc4-b581-111b0afaf43f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.884763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d621d-c19a-4f47-947e-77a8bf5585fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.885529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d621d-c19a-4f47-947e-77a8bf5585fc admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.885989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf10d-fd68-4190-a4f5-a31f6e8e437d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.886104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d621d-c19a-4f47-947e-77a8bf5585fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.886383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d621d-c19a-4f47-947e-77a8bf5585fc admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.887337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d621d-c19a-4f47-947e-77a8bf5585fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.889589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d621d-c19a-4f47-947e-77a8bf5585fc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.896039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a87e6-0c2a-4e9c-a004-9e02c56d05df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.896670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a87e6-0c2a-4e9c-a004-9e02c56d05df admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.896927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a87e6-0c2a-4e9c-a004-9e02c56d05df admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.897232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a87e6-0c2a-4e9c-a004-9e02c56d05df admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.897861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a87e6-0c2a-4e9c-a004-9e02c56d05df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.899801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a87e6-0c2a-4e9c-a004-9e02c56d05df admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.916812 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1036/2093] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:06.920510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d621d-c19a-4f47-947e-77a8bf5585fc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.930409 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1058/2094] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:06 2022] POST /identity/v3/users => generated 310 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:06.931584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a87e6-0c2a-4e9c-a004-9e02c56d05df admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.932049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d621d-c19a-4f47-947e-77a8bf5585fc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.937429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaadb3-e1fe-43f2-b684-0011920d9c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.939035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaadb3-e1fe-43f2-b684-0011920d9c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.939400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaadb3-e1fe-43f2-b684-0011920d9c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaadb3-e1fe-43f2-b684-0011920d9c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.940608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baaadb3-e1fe-43f2-b684-0011920d9c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.942671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a87e6-0c2a-4e9c-a004-9e02c56d05df admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.943041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baaadb3-e1fe-43f2-b684-0011920d9c2c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.948168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cc9e0-846e-43be-9a68-60c4660ccdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.948775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cc9e0-846e-43be-9a68-60c4660ccdee admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.949061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cc9e0-846e-43be-9a68-60c4660ccdee admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.949659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cc9e0-846e-43be-9a68-60c4660ccdee admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.950002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cc9e0-846e-43be-9a68-60c4660ccdee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.951639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cc9e0-846e-43be-9a68-60c4660ccdee admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:06.956074 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1037/2095] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:06.960794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaadb3-e1fe-43f2-b684-0011920d9c2c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.966314 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1059/2096] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:06.966470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:06.967527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cc9e0-846e-43be-9a68-60c4660ccdee admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:06.980037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cc9e0-846e-43be-9a68-60c4660ccdee admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.980567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaadb3-e1fe-43f2-b684-0011920d9c2c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:06.985868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9feae-0aae-4c2b-be06-8b32faa10b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.986786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9feae-0aae-4c2b-be06-8b32faa10b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.987221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9feae-0aae-4c2b-be06-8b32faa10b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.987584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9feae-0aae-4c2b-be06-8b32faa10b61 admin admin] PATH_INFO: `/v3/projects/e2fa947d1aac49f4b3dde35d21b56370/users/bc56793fe99c45afb8fc23fd5f2c342f/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.992208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddde4fe-db1b-42a6-90e8-375d8e47c733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:06.993761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde4fe-db1b-42a6-90e8-375d8e47c733 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:06.994494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde4fe-db1b-42a6-90e8-375d8e47c733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:06.994843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde4fe-db1b-42a6-90e8-375d8e47c733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:06.995577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddde4fe-db1b-42a6-90e8-375d8e47c733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:06.997271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddde4fe-db1b-42a6-90e8-375d8e47c733 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:07.006790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9feae-0aae-4c2b-be06-8b32faa10b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2fa947d1aac49f4b3dde35d21b56370, user_id=bc56793fe99c45afb8fc23fd5f2c342f, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:07.009859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9feae-0aae-4c2b-be06-8b32faa10b61 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:07.011670 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1038/2097] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:07.014210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddde4fe-db1b-42a6-90e8-375d8e47c733 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:07.032475 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1060/2098] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:06 2022] PUT /identity/v3/projects/e2fa947d1aac49f4b3dde35d21b56370/users/bc56793fe99c45afb8fc23fd5f2c342f/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:07.037216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9feae-0aae-4c2b-be06-8b32faa10b61 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:07.040806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddde4fe-db1b-42a6-90e8-375d8e47c733 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:07.130715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:07.204545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9feae-0aae-4c2b-be06-8b32faa10b61 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:07.216389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307cc46-a4d9-49d7-b26a-232742f9261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:07.217189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307cc46-a4d9-49d7-b26a-232742f9261f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:07.217481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307cc46-a4d9-49d7-b26a-232742f9261f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:07.217826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307cc46-a4d9-49d7-b26a-232742f9261f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:07.218564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307cc46-a4d9-49d7-b26a-232742f9261f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:07.220455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307cc46-a4d9-49d7-b26a-232742f9261f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:07.224596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1a49b-8c26-4686-9968-342119454ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:07.225330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1a49b-8c26-4686-9968-342119454ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:07.225537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1a49b-8c26-4686-9968-342119454ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:07.225826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1a49b-8c26-4686-9968-342119454ba3 admin admin] PATH_INFO: `/v3/projects/9421f2a8299540af9c5dfca2965c578f/users/cd49cd7c67754f1fbdb1453478ef23a9/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:07.245338 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1061/2099] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:07.247279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307cc46-a4d9-49d7-b26a-232742f9261f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:07.250844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1a49b-8c26-4686-9968-342119454ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9421f2a8299540af9c5dfca2965c578f, user_id=cd49cd7c67754f1fbdb1453478ef23a9, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:07.253719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd1a49b-8c26-4686-9968-342119454ba3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:07.262026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307cc46-a4d9-49d7-b26a-232742f9261f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:07.276338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1039/2100] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:07 2022] PUT /identity/v3/projects/9421f2a8299540af9c5dfca2965c578f/users/cd49cd7c67754f1fbdb1453478ef23a9/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:07.277880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18872819-b3d4-49a7-88a6-0efd871b3d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:07.277880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18872819-b3d4-49a7-88a6-0efd871b3d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:07.278148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18872819-b3d4-49a7-88a6-0efd871b3d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:07.278428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18872819-b3d4-49a7-88a6-0efd871b3d29 admin admin] PATH_INFO: `/v3/projects/e2fa947d1aac49f4b3dde35d21b56370/users/bc56793fe99c45afb8fc23fd5f2c342f/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:07.280707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18872819-b3d4-49a7-88a6-0efd871b3d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2fa947d1aac49f4b3dde35d21b56370, user_id=bc56793fe99c45afb8fc23fd5f2c342f, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:07.283333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18872819-b3d4-49a7-88a6-0efd871b3d29 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:07.288771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e7a72-fadb-4e03-bfe5-2ac5b3fadcad None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:07.289059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e7a72-fadb-4e03-bfe5-2ac5b3fadcad None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:07.289496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e7a72-fadb-4e03-bfe5-2ac5b3fadcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:07.312804 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1062/2101] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:07 2022] PUT /identity/v3/projects/e2fa947d1aac49f4b3dde35d21b56370/users/bc56793fe99c45afb8fc23fd5f2c342f/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:07.324939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1754321-20d1-4f8b-bd8b-40f55fc4cd1f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:07.325225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1754321-20d1-4f8b-bd8b-40f55fc4cd1f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:07.325669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1754321-20d1-4f8b-bd8b-40f55fc4cd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:07.337089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-376e7a72-fadb-4e03-bfe5-2ac5b3fadcad None None] MFA Rules not processed for user `cd49cd7c67754f1fbdb1453478ef23a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:07.338495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-376e7a72-fadb-4e03-bfe5-2ac5b3fadcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:07.360880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b1754321-20d1-4f8b-bd8b-40f55fc4cd1f None None] MFA Rules not processed for user `bc56793fe99c45afb8fc23fd5f2c342f`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:07.362644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b1754321-20d1-4f8b-bd8b-40f55fc4cd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:07.442833 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1040/2102] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:07.456819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e7a72-fadb-4e03-bfe5-2ac5b3fadcad None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:07.491421 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1063/2103] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:07.502789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1754321-20d1-4f8b-bd8b-40f55fc4cd1f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:07.505068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b1754321-20d1-4f8b-bd8b-40f55fc4cd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:07.531925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e7a72-fadb-4e03-bfe5-2ac5b3fadcad None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:07.535776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f59cfb-34bc-439f-b2cc-40fc028d7d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:07.536989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f59cfb-34bc-439f-b2cc-40fc028d7d57 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:07.537272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f59cfb-34bc-439f-b2cc-40fc028d7d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:07.537544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f59cfb-34bc-439f-b2cc-40fc028d7d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:07.546139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f59cfb-34bc-439f-b2cc-40fc028d7d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:07.547592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f59cfb-34bc-439f-b2cc-40fc028d7d57 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:07.558588 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1041/2104] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:07.577011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1754321-20d1-4f8b-bd8b-40f55fc4cd1f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:07.581498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989687d3-3634-4c4b-9d2f-1d42c8b32d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:07.582279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989687d3-3634-4c4b-9d2f-1d42c8b32d5d service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:07.583214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989687d3-3634-4c4b-9d2f-1d42c8b32d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:07.583739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989687d3-3634-4c4b-9d2f-1d42c8b32d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:07.595834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989687d3-3634-4c4b-9d2f-1d42c8b32d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:07.597139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989687d3-3634-4c4b-9d2f-1d42c8b32d5d service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:07.611069 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1064/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:07 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:07.854983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f59cfb-34bc-439f-b2cc-40fc028d7d57 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:07.874588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f59cfb-34bc-439f-b2cc-40fc028d7d57 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:07.880911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb5b61-00eb-4095-bd50-e20633867424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:07.881754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5b61-00eb-4095-bd50-e20633867424 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:07.882021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5b61-00eb-4095-bd50-e20633867424 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:07.882295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5b61-00eb-4095-bd50-e20633867424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:07.887342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:07.978887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb5b61-00eb-4095-bd50-e20633867424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:07.980462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb5b61-00eb-4095-bd50-e20633867424 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:07.993052 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1042/2106] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:07 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:08.990501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989687d3-3634-4c4b-9d2f-1d42c8b32d5d service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:09.048005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:09.075430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989687d3-3634-4c4b-9d2f-1d42c8b32d5d service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:09.080884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:09.081544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:09.081924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:09.082548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] PATH_INFO: `/v3/users/2fec55ae6e2e46deaaa74c1d0b20d11e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:09.084127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fec55ae6e2e46deaaa74c1d0b20d11e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:09.086211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:09.149891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fec55ae6e2e46deaaa74c1d0b20d11e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:09.158085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fec55ae6e2e46deaaa74c1d0b20d11e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:09.159084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fec55ae6e2e46deaaa74c1d0b20d11e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:09.159793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fec55ae6e2e46deaaa74c1d0b20d11e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:09.169275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fec55ae6e2e46deaaa74c1d0b20d11e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:09.196876 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1065/2107] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:08 2022] DELETE /identity/v3/users/2fec55ae6e2e46deaaa74c1d0b20d11e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:09.201872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5d367-8bcf-4125-aae5-8a23985f8e90 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:09.209613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5d367-8bcf-4125-aae5-8a23985f8e90 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:09.210124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5d367-8bcf-4125-aae5-8a23985f8e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:09.244806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cea5d367-8bcf-4125-aae5-8a23985f8e90 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:09.246150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cea5d367-8bcf-4125-aae5-8a23985f8e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:09.330799 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1043/2108] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:09.341184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:09.342664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:09.409992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486735a2-f7f3-4de4-a97e-b469a9f24419 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:09.418405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17772976-c7b8-437a-add9-344e24fb190c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:09.419043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17772976-c7b8-437a-add9-344e24fb190c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:09.419323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17772976-c7b8-437a-add9-344e24fb190c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:09.420243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17772976-c7b8-437a-add9-344e24fb190c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:09.433324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17772976-c7b8-437a-add9-344e24fb190c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:09.436347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17772976-c7b8-437a-add9-344e24fb190c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:09.457299 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1066/2109] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:09.937673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5d367-8bcf-4125-aae5-8a23985f8e90 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:09.939009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cea5d367-8bcf-4125-aae5-8a23985f8e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:09.959245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5d367-8bcf-4125-aae5-8a23985f8e90 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:09.964562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:09.965496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:09.965990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:09.966275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] PATH_INFO: `/v3/projects/93a2bc6de070450da7993395b38458ed` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:09.967496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a2bc6de070450da7993395b38458ed)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:09.969725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:10.032578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a2bc6de070450da7993395b38458ed', 'request_id': 'req-3881267e-1dc7-4728-8221-16cf9ed8da88'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.043428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a2bc6de070450da7993395b38458ed', 'request_id': 'req-3881267e-1dc7-4728-8221-16cf9ed8da88'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.051226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a2bc6de070450da7993395b38458ed', 'request_id': 'req-3881267e-1dc7-4728-8221-16cf9ed8da88'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.064372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] The token cache is being invalidate because project 93a2bc6de070450da7993395b38458ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:10.064762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.065564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.068474 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1044/2110] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:09 2022] DELETE /identity/v3/projects/93a2bc6de070450da7993395b38458ed => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:10.073133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17772976-c7b8-437a-add9-344e24fb190c service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:10.088047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17772976-c7b8-437a-add9-344e24fb190c service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:10.181969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:10.182872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:10.183152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:10.183425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] PATH_INFO: `/v3/users/566d9b1b65994649b969cf96be8de89e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:10.184680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=566d9b1b65994649b969cf96be8de89e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:10.187040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:10.190648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:10.249302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566d9b1b65994649b969cf96be8de89e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.265501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566d9b1b65994649b969cf96be8de89e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.266545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566d9b1b65994649b969cf96be8de89e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.267862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566d9b1b65994649b969cf96be8de89e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.278980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fe0c78ef-5f54-42e7-aa03-84e8fe96edb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566d9b1b65994649b969cf96be8de89e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:10.307201 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1067/2111] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:10 2022] DELETE /identity/v3/users/566d9b1b65994649b969cf96be8de89e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:10.834447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:10.836324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:10.867738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881267e-1dc7-4728-8221-16cf9ed8da88 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:10.941050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:11.010958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:11.019303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.021226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.022095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] PATH_INFO: `/v3/projects/eae8bcadd2b0420ea88bd8ae51c3f01b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.024751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae8bcadd2b0420ea88bd8ae51c3f01b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:11.027473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:11.106566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae8bcadd2b0420ea88bd8ae51c3f01b', 'request_id': 'req-c25df202-a457-4943-bace-d8cd12a3788a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:11.119934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae8bcadd2b0420ea88bd8ae51c3f01b', 'request_id': 'req-c25df202-a457-4943-bace-d8cd12a3788a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:11.131182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae8bcadd2b0420ea88bd8ae51c3f01b', 'request_id': 'req-c25df202-a457-4943-bace-d8cd12a3788a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:11.147273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] The token cache is being invalidate because project eae8bcadd2b0420ea88bd8ae51c3f01b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:11.147575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:11.148499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c25df202-a457-4943-bace-d8cd12a3788a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:11.151388 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1045/2112] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:10 2022] DELETE /identity/v3/projects/eae8bcadd2b0420ea88bd8ae51c3f01b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:11.156669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab8f8-df8a-4ebe-a857-74b7dfa1ca4f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.158081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab8f8-df8a-4ebe-a857-74b7dfa1ca4f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.158472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab8f8-df8a-4ebe-a857-74b7dfa1ca4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.182891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1c9ab8f8-df8a-4ebe-a857-74b7dfa1ca4f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:11.184580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1c9ab8f8-df8a-4ebe-a857-74b7dfa1ca4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:11.329451 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1068/2113] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:11.338617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ade6b4-aaf0-4931-a201-a843db7d8426 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.338899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ade6b4-aaf0-4931-a201-a843db7d8426 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.339171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ade6b4-aaf0-4931-a201-a843db7d8426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.373244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-15ade6b4-aaf0-4931-a201-a843db7d8426 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:11.374497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-15ade6b4-aaf0-4931-a201-a843db7d8426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:11.400595 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1046/2114] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:11.404596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab8f8-df8a-4ebe-a857-74b7dfa1ca4f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:11.418026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab8f8-df8a-4ebe-a857-74b7dfa1ca4f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:11.430457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:11.431471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.432013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.432496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.433915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:11.434458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:11.438779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:11.453917 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1069/2115] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:37:11.461143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ade6b4-aaf0-4931-a201-a843db7d8426 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:11.471940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ade6b4-aaf0-4931-a201-a843db7d8426 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:11.488811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a741777-4c09-4b85-bb8e-d9efba0b2256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:11.489562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a741777-4c09-4b85-bb8e-d9efba0b2256 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.489918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a741777-4c09-4b85-bb8e-d9efba0b2256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.490149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a741777-4c09-4b85-bb8e-d9efba0b2256 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.490876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a741777-4c09-4b85-bb8e-d9efba0b2256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:11.493322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a741777-4c09-4b85-bb8e-d9efba0b2256 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:11.540433 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1047/2116] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:11 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:11.544418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:11.555428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e1178-50c4-4b80-b784-ed6085b9748e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:11.640926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c9fdc-f993-437b-8324-80d9f8a861e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:11.641572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c9fdc-f993-437b-8324-80d9f8a861e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.642346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c9fdc-f993-437b-8324-80d9f8a861e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.642346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c9fdc-f993-437b-8324-80d9f8a861e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.642961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c9fdc-f993-437b-8324-80d9f8a861e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:11.645344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c9fdc-f993-437b-8324-80d9f8a861e9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:11.663866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:11.674986 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1070/2117] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:11 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:11.679505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a741777-4c09-4b85-bb8e-d9efba0b2256 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:11.696493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a741777-4c09-4b85-bb8e-d9efba0b2256 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:11.700982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cd39a-69fb-4933-a9b4-ececc0e9d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:11.701616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cd39a-69fb-4933-a9b4-ececc0e9d528 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.702033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cd39a-69fb-4933-a9b4-ececc0e9d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.702261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cd39a-69fb-4933-a9b4-ececc0e9d528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.702923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cd39a-69fb-4933-a9b4-ececc0e9d528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:11.704525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cd39a-69fb-4933-a9b4-ececc0e9d528 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:11.716574 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1048/2118] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:11.722905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c9fdc-f993-437b-8324-80d9f8a861e9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:11.742978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c9fdc-f993-437b-8324-80d9f8a861e9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:11.748613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d464231-72ec-4cf9-8999-ce5ba8c450e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:11.750002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464231-72ec-4cf9-8999-ce5ba8c450e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.750643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464231-72ec-4cf9-8999-ce5ba8c450e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.751164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464231-72ec-4cf9-8999-ce5ba8c450e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.752692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d464231-72ec-4cf9-8999-ce5ba8c450e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:11.757976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d464231-72ec-4cf9-8999-ce5ba8c450e3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:11.779467 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1071/2119] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:11.783169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cd39a-69fb-4933-a9b4-ececc0e9d528 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:11.805180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cd39a-69fb-4933-a9b4-ececc0e9d528 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:11.822160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4bac3-8242-4523-bff0-412fccd086fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:11.823337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4bac3-8242-4523-bff0-412fccd086fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.824376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4bac3-8242-4523-bff0-412fccd086fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.824777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4bac3-8242-4523-bff0-412fccd086fe admin admin] PATH_INFO: `/v3/projects/054907db3740425db8d593526f212875/users/747b96fc08c24b5c83d8efc239b2e39c/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.851306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4bac3-8242-4523-bff0-412fccd086fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=054907db3740425db8d593526f212875, user_id=747b96fc08c24b5c83d8efc239b2e39c, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:11.854498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4bac3-8242-4523-bff0-412fccd086fe admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:11.873270 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1049/2120] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:11 2022] PUT /identity/v3/projects/054907db3740425db8d593526f212875/users/747b96fc08c24b5c83d8efc239b2e39c/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:11.885314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc7ce-71de-49f1-be5d-78952ca3dc16 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:11.885689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc7ce-71de-49f1-be5d-78952ca3dc16 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:11.885946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc7ce-71de-49f1-be5d-78952ca3dc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:11.951035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cdbcc7ce-71de-49f1-be5d-78952ca3dc16 None None] MFA Rules not processed for user `747b96fc08c24b5c83d8efc239b2e39c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:11.952322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cdbcc7ce-71de-49f1-be5d-78952ca3dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:12.030662 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1072/2121] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:12.038234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4bac3-8242-4523-bff0-412fccd086fe admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.125179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:12.133942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4bac3-8242-4523-bff0-412fccd086fe admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:12.141305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a4a5-92ef-4ec4-ad77-8b00a4881ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:12.142208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a4a5-92ef-4ec4-ad77-8b00a4881ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.142699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a4a5-92ef-4ec4-ad77-8b00a4881ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.142972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a4a5-92ef-4ec4-ad77-8b00a4881ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.143621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025a4a5-92ef-4ec4-ad77-8b00a4881ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:12.146836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025a4a5-92ef-4ec4-ad77-8b00a4881ce0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:12.181268 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1050/2122] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:12 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:12.185276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc7ce-71de-49f1-be5d-78952ca3dc16 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.210610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc7ce-71de-49f1-be5d-78952ca3dc16 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:12.274915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d2c14-bca0-4018-bca3-7e55133a362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:12.275449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d2c14-bca0-4018-bca3-7e55133a362a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.275724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d2c14-bca0-4018-bca3-7e55133a362a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.275998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d2c14-bca0-4018-bca3-7e55133a362a admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.276629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d2c14-bca0-4018-bca3-7e55133a362a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:12.278605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d2c14-bca0-4018-bca3-7e55133a362a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:12.306796 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1073/2123] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:12 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:12.310845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a4a5-92ef-4ec4-ad77-8b00a4881ce0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.321537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a4a5-92ef-4ec4-ad77-8b00a4881ce0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:12.326759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e068ba-2e4f-44a7-846b-20461c573fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:12.327373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e068ba-2e4f-44a7-846b-20461c573fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.327682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e068ba-2e4f-44a7-846b-20461c573fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.327959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e068ba-2e4f-44a7-846b-20461c573fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.328610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e068ba-2e4f-44a7-846b-20461c573fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:12.330239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e068ba-2e4f-44a7-846b-20461c573fb9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:12.342333 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1051/2124] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:12.349022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d2c14-bca0-4018-bca3-7e55133a362a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.358727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d2c14-bca0-4018-bca3-7e55133a362a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:12.364475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64660a57-0e19-429e-b971-c3973e62d2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:12.365274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64660a57-0e19-429e-b971-c3973e62d2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.365561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64660a57-0e19-429e-b971-c3973e62d2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.365871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64660a57-0e19-429e-b971-c3973e62d2e5 admin admin] PATH_INFO: `/v3/projects/0005547a8f284488a8876ef1614cd0e2/users/a1f971f041d54ef684521814774fcd19/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.377792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15beae0-2bde-4642-894c-63df6fe4f9dd None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.378050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15beae0-2bde-4642-894c-63df6fe4f9dd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.378486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15beae0-2bde-4642-894c-63df6fe4f9dd None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.381044 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1052/2125] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:37:12.381870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64660a57-0e19-429e-b971-c3973e62d2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0005547a8f284488a8876ef1614cd0e2, user_id=a1f971f041d54ef684521814774fcd19, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:12.384804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64660a57-0e19-429e-b971-c3973e62d2e5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:12.406574 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1074/2126] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:12 2022] PUT /identity/v3/projects/0005547a8f284488a8876ef1614cd0e2/users/a1f971f041d54ef684521814774fcd19/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:12.410317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64660a57-0e19-429e-b971-c3973e62d2e5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.411770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b324b8-08e4-4c70-a7da-e0554113bc8b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.412262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b324b8-08e4-4c70-a7da-e0554113bc8b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.412489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b324b8-08e4-4c70-a7da-e0554113bc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.446855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e6b324b8-08e4-4c70-a7da-e0554113bc8b None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:12.448083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e6b324b8-08e4-4c70-a7da-e0554113bc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:12.476810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:12.497173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64660a57-0e19-429e-b971-c3973e62d2e5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:12.502447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:12.503054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.503339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.503614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.504797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:12.506884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:12.516860 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1075/2127] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:12.517692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.519271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:12.522326 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1053/2128] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:37:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:12.523137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b324b8-08e4-4c70-a7da-e0554113bc8b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.533462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9b90e-ee6f-4784-8338-dc4caf34e043 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:12.535072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b324b8-08e4-4c70-a7da-e0554113bc8b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:12.539126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:12.539764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.540051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.540399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] PATH_INFO: `/v3/users/c75799eec4744849a52d589e19424d6e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.541524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75799eec4744849a52d589e19424d6e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:12.546840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:12.564078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:12.564778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.565057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.565327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] PATH_INFO: `/v3/projects/0005547a8f284488a8876ef1614cd0e2/users/a1f971f041d54ef684521814774fcd19/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.570139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0005547a8f284488a8876ef1614cd0e2, user_id=a1f971f041d54ef684521814774fcd19, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:12.575760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:12.607395 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1054/2129] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:12 2022] PUT /identity/v3/projects/0005547a8f284488a8876ef1614cd0e2/users/a1f971f041d54ef684521814774fcd19/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:12.611752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.618473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:12.632581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75799eec4744849a52d589e19424d6e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:12.640822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75799eec4744849a52d589e19424d6e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:12.641773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75799eec4744849a52d589e19424d6e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:12.642394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75799eec4744849a52d589e19424d6e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:12.657218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e548e7f3-1b9c-413f-9849-968cb70a97e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75799eec4744849a52d589e19424d6e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:12.714413 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1076/2130] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:12 2022] DELETE /identity/v3/users/c75799eec4744849a52d589e19424d6e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:12.715472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1aaf-902f-4594-84cc-9d1c219ccac3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.715752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1aaf-902f-4594-84cc-9d1c219ccac3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.716031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1aaf-902f-4594-84cc-9d1c219ccac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.772038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e52e08-2223-4cee-b39c-ce0b19e69f62 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:12.795122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b38d1aaf-902f-4594-84cc-9d1c219ccac3 None None] MFA Rules not processed for user `a1f971f041d54ef684521814774fcd19`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:12.796344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b38d1aaf-902f-4594-84cc-9d1c219ccac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:12.847521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:12.863507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:12.864815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:12.864930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:12.865246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:12.869036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:12.884545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:12.886455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:12.906393 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1055/2131] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:12.921425 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1077/2132] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:12.932162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.935015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:12.976788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1aaf-902f-4594-84cc-9d1c219ccac3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:12.979838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b38d1aaf-902f-4594-84cc-9d1c219ccac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:13.039550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:13.040035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b086d-3a64-486c-9b2f-5811cb51c41a service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.044787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747f092-8433-439f-ba1a-6fe1bc65dd3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.045595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f092-8433-439f-ba1a-6fe1bc65dd3e service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.046584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f092-8433-439f-ba1a-6fe1bc65dd3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.046933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f092-8433-439f-ba1a-6fe1bc65dd3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.062758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747f092-8433-439f-ba1a-6fe1bc65dd3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.065213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747f092-8433-439f-ba1a-6fe1bc65dd3e service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.076891 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1056/2133] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:13.080123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1aaf-902f-4594-84cc-9d1c219ccac3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.086833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.087519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.087952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.088235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] PATH_INFO: `/v3/projects/732bb88e765b4fc3921cb3e37be0270a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.089600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=732bb88e765b4fc3921cb3e37be0270a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.091899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.166596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '732bb88e765b4fc3921cb3e37be0270a', 'request_id': 'req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:13.184084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '732bb88e765b4fc3921cb3e37be0270a', 'request_id': 'req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:13.193144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '732bb88e765b4fc3921cb3e37be0270a', 'request_id': 'req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:13.205439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] The token cache is being invalidate because project 732bb88e765b4fc3921cb3e37be0270a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:13.205897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:13.207257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9d59d3b9-f03f-4cd8-8f55-53cec7691fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:13.210162 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1078/2134] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:12 2022] DELETE /identity/v3/projects/732bb88e765b4fc3921cb3e37be0270a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:13.216074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d820-b63b-4b24-91e1-518212c6000b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.216367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d820-b63b-4b24-91e1-518212c6000b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.216647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d820-b63b-4b24-91e1-518212c6000b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.281982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a041d820-b63b-4b24-91e1-518212c6000b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:13.283310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a041d820-b63b-4b24-91e1-518212c6000b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:13.375117 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1057/2135] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:13.384643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c085-229c-4091-9221-c8a7adc76913 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.385158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c085-229c-4091-9221-c8a7adc76913 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.385565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89c085-229c-4091-9221-c8a7adc76913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.409275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dc89c085-229c-4091-9221-c8a7adc76913 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:13.411480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dc89c085-229c-4091-9221-c8a7adc76913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:13.420387 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1079/2136] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:13.424648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041d820-b63b-4b24-91e1-518212c6000b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.435951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041d820-b63b-4b24-91e1-518212c6000b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.440679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.441287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.441565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.441881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.442610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:13.442917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.444503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.458664 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1058/2137] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:13.461246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89c085-229c-4091-9221-c8a7adc76913 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.481331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89c085-229c-4091-9221-c8a7adc76913 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.486218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098de9b-9102-4df3-8621-35007345b7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.486952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098de9b-9102-4df3-8621-35007345b7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.487305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098de9b-9102-4df3-8621-35007345b7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.487636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098de9b-9102-4df3-8621-35007345b7f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.488328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098de9b-9102-4df3-8621-35007345b7f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.491577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098de9b-9102-4df3-8621-35007345b7f7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.507567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.509617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:13.529161 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1080/2138] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:13 2022] POST /identity/v3/projects => generated 391 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:13.533543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098de9b-9102-4df3-8621-35007345b7f7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.579852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1046f-0b21-43df-8384-4f2eb0f293ed admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.599950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:13.602552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098de9b-9102-4df3-8621-35007345b7f7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.608281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af20a88-ae3b-4171-b83c-96d9b8298354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.608895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af20a88-ae3b-4171-b83c-96d9b8298354 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.609177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af20a88-ae3b-4171-b83c-96d9b8298354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.609495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af20a88-ae3b-4171-b83c-96d9b8298354 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.610217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af20a88-ae3b-4171-b83c-96d9b8298354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.612209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af20a88-ae3b-4171-b83c-96d9b8298354 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.642877 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1081/2139] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:13 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:13.644359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af20a88-ae3b-4171-b83c-96d9b8298354 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.650151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f971f041d54ef684521814774fcd19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0005547a8f284488a8876ef1614cd0e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.650856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.651493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.651628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.652250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:13.652590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.654457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.664145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af20a88-ae3b-4171-b83c-96d9b8298354 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.669675 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1059/2140] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:37:13.669849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd91ec-0dd7-4280-852f-b9f739a530d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.670332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd91ec-0dd7-4280-852f-b9f739a530d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.670611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd91ec-0dd7-4280-852f-b9f739a530d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.670874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd91ec-0dd7-4280-852f-b9f739a530d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.671646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd91ec-0dd7-4280-852f-b9f739a530d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.672550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.673249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd91ec-0dd7-4280-852f-b9f739a530d5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.682841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cd5c2-1524-401f-88ad-4924f9ca11d2 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.684009 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1082/2141] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:13.688405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd91ec-0dd7-4280-852f-b9f739a530d5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.689786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038392ea-d54e-48a8-88f5-9d50777791e0 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f971f041d54ef684521814774fcd19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0005547a8f284488a8876ef1614cd0e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.690492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038392ea-d54e-48a8-88f5-9d50777791e0 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.690784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038392ea-d54e-48a8-88f5-9d50777791e0 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.691062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038392ea-d54e-48a8-88f5-9d50777791e0 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.691798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038392ea-d54e-48a8-88f5-9d50777791e0 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.694354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038392ea-d54e-48a8-88f5-9d50777791e0 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.698359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd91ec-0dd7-4280-852f-b9f739a530d5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.703669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6cf87-dbde-4256-bd03-5a3e58aa611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.704820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6cf87-dbde-4256-bd03-5a3e58aa611a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.705187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6cf87-dbde-4256-bd03-5a3e58aa611a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.706189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6cf87-dbde-4256-bd03-5a3e58aa611a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.706907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6cf87-dbde-4256-bd03-5a3e58aa611a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.710010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6cf87-dbde-4256-bd03-5a3e58aa611a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.722199 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1083/2142] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:13.727128 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1060/2143] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:13 2022] POST /identity/v3/projects => generated 354 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:13.729720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6cf87-dbde-4256-bd03-5a3e58aa611a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.731177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038392ea-d54e-48a8-88f5-9d50777791e0 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.736925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:13.744015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6cf87-dbde-4256-bd03-5a3e58aa611a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.800693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038392ea-d54e-48a8-88f5-9d50777791e0 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.807168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fca93-ae97-4585-99c9-ca34c3096e37 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f971f041d54ef684521814774fcd19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0005547a8f284488a8876ef1614cd0e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.807860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fca93-ae97-4585-99c9-ca34c3096e37 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.808219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fca93-ae97-4585-99c9-ca34c3096e37 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.808696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fca93-ae97-4585-99c9-ca34c3096e37 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.809301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fca93-ae97-4585-99c9-ca34c3096e37 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:13.809708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fca93-ae97-4585-99c9-ca34c3096e37 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.809854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b26ee-de16-4db3-be1b-c6932e80cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.810515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b26ee-de16-4db3-be1b-c6932e80cdac admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.810880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b26ee-de16-4db3-be1b-c6932e80cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.811214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b26ee-de16-4db3-be1b-c6932e80cdac admin admin] PATH_INFO: `/v3/projects/4290a542e73045309f6ceb8d4f5a267b/users/7e04471e8c634a6586789f82204241a7/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.811686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fca93-ae97-4585-99c9-ca34c3096e37 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.825673 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1061/2144] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:37:13.832645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b26ee-de16-4db3-be1b-c6932e80cdac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4290a542e73045309f6ceb8d4f5a267b, user_id=7e04471e8c634a6586789f82204241a7, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.835383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b26ee-de16-4db3-be1b-c6932e80cdac admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:13.857786 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1084/2145] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:13 2022] PUT /identity/v3/projects/4290a542e73045309f6ceb8d4f5a267b/users/7e04471e8c634a6586789f82204241a7/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:13.861797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a1135-7b95-46fb-8aeb-01e97c0be2b4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.862022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a1135-7b95-46fb-8aeb-01e97c0be2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.862346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a1135-7b95-46fb-8aeb-01e97c0be2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.875194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b26ee-de16-4db3-be1b-c6932e80cdac admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:13.892852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-097a1135-7b95-46fb-8aeb-01e97c0be2b4 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:13.894247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-097a1135-7b95-46fb-8aeb-01e97c0be2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:13.981870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b26ee-de16-4db3-be1b-c6932e80cdac admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:13.988278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1f971f041d54ef684521814774fcd19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0005547a8f284488a8876ef1614cd0e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:13.989783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:13.990099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:13.991153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] PATH_INFO: `/v3/projects/e46786ceeda34d2bba21635179e303b9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:13.992956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e46786ceeda34d2bba21635179e303b9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:13.995321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:14.023471 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1062/2146] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:14.033064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a1135-7b95-46fb-8aeb-01e97c0be2b4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:14.036196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-097a1135-7b95-46fb-8aeb-01e97c0be2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:14.102863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46786ceeda34d2bba21635179e303b9', 'request_id': 'req-bf57b926-ea2b-4657-8123-9a6599aba396'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.107511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:14.114100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46786ceeda34d2bba21635179e303b9', 'request_id': 'req-bf57b926-ea2b-4657-8123-9a6599aba396'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.121757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46786ceeda34d2bba21635179e303b9', 'request_id': 'req-bf57b926-ea2b-4657-8123-9a6599aba396'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.134389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] The token cache is being invalidate because project e46786ceeda34d2bba21635179e303b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:14.134702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.135358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.137883 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1085/2147] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:13 2022] DELETE /identity/v3/projects/e46786ceeda34d2bba21635179e303b9 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:14.138438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:14.139721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:14.144184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a1135-7b95-46fb-8aeb-01e97c0be2b4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:14.174321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:14.231576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e7e7a-7fca-45b4-971d-593ee7da405f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:14.232703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e7e7a-7fca-45b4-971d-593ee7da405f service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:14.233187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e7e7a-7fca-45b4-971d-593ee7da405f service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:14.233655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e7e7a-7fca-45b4-971d-593ee7da405f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:14.236040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-107e7e7a-7fca-45b4-971d-593ee7da405f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:14.250372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57b926-ea2b-4657-8123-9a6599aba396 tempest-VolumeQuotaClassesTest-1181322110 tempest-VolumeQuotaClassesTest-1181322110-project-admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:14.250917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e7e7a-7fca-45b4-971d-593ee7da405f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:14.252162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e7e7a-7fca-45b4-971d-593ee7da405f service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:14.265233 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1063/2148] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:14.266027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:14.267469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:14.267469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:14.267713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] PATH_INFO: `/v3/users/e905de725f9749319651c6eb2a1df8c0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:14.269265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e905de725f9749319651c6eb2a1df8c0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:14.271627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:14.335417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e905de725f9749319651c6eb2a1df8c0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.347384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e905de725f9749319651c6eb2a1df8c0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.348613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e905de725f9749319651c6eb2a1df8c0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.349365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e905de725f9749319651c6eb2a1df8c0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.360185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e905de725f9749319651c6eb2a1df8c0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.389736 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1086/2149] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:14 2022] DELETE /identity/v3/users/e905de725f9749319651c6eb2a1df8c0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:14.392362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766858-3517-4583-b1b1-4d44a20b0793 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:14.392653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766858-3517-4583-b1b1-4d44a20b0793 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:14.392931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766858-3517-4583-b1b1-4d44a20b0793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:14.429334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-be766858-3517-4583-b1b1-4d44a20b0793 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:14.430929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be766858-3517-4583-b1b1-4d44a20b0793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:14.452767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:14.454216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:14.574816 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1064/2150] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:14.585444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be766858-3517-4583-b1b1-4d44a20b0793 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:14.587663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be766858-3517-4583-b1b1-4d44a20b0793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:14.610125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bb68e-3553-4ccb-913e-f2c0bdfb36f1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:14.616943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:14.617608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:14.617917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:14.618192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] PATH_INFO: `/v3/users/747b96fc08c24b5c83d8efc239b2e39c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:14.619961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] RBAC: Authorizing `identity:delete_user(user_id=747b96fc08c24b5c83d8efc239b2e39c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:14.622047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:14.667446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be766858-3517-4583-b1b1-4d44a20b0793 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:14.675114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57621619-1a6b-4846-ab9a-7d5155bb5e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:14.675896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57621619-1a6b-4846-ab9a-7d5155bb5e0d service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:14.676114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57621619-1a6b-4846-ab9a-7d5155bb5e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:14.676994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57621619-1a6b-4846-ab9a-7d5155bb5e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:14.678764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:14.684150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '747b96fc08c24b5c83d8efc239b2e39c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.689035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57621619-1a6b-4846-ab9a-7d5155bb5e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:14.690708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57621619-1a6b-4846-ab9a-7d5155bb5e0d service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:14.700327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '747b96fc08c24b5c83d8efc239b2e39c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.701684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '747b96fc08c24b5c83d8efc239b2e39c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.702742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '747b96fc08c24b5c83d8efc239b2e39c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.706494 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1065/2151] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:14.714708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '747b96fc08c24b5c83d8efc239b2e39c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:14.743630 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1087/2152] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:14 2022] DELETE /identity/v3/users/747b96fc08c24b5c83d8efc239b2e39c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:14.748208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abe6b9-8b17-4c00-9b7f-bc53a91bb48b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:14.748488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abe6b9-8b17-4c00-9b7f-bc53a91bb48b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:14.748811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abe6b9-8b17-4c00-9b7f-bc53a91bb48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:14.779000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-74abe6b9-8b17-4c00-9b7f-bc53a91bb48b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:14.780227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-74abe6b9-8b17-4c00-9b7f-bc53a91bb48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:14.814309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:14.872753 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1066/2153] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:14.903404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:14.905731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:14.999679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b435f-cd54-4385-9cb8-f9731624a08f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:15.006483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244e729-aa9f-4214-b1d8-e523190bd1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:15.006483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244e729-aa9f-4214-b1d8-e523190bd1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:15.006785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244e729-aa9f-4214-b1d8-e523190bd1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:15.007013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244e729-aa9f-4214-b1d8-e523190bd1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:15.018403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244e729-aa9f-4214-b1d8-e523190bd1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:15.019950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244e729-aa9f-4214-b1d8-e523190bd1d4 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:15.032679 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1088/2154] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:15.112894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abe6b9-8b17-4c00-9b7f-bc53a91bb48b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:15.115099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-74abe6b9-8b17-4c00-9b7f-bc53a91bb48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:15.132871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abe6b9-8b17-4c00-9b7f-bc53a91bb48b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:15.139122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:15.140318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:15.140801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:15.141299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] PATH_INFO: `/v3/projects/bf26bfcc5c98442ab5af3717927f7038` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:15.142798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf26bfcc5c98442ab5af3717927f7038)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:15.146880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:15.244271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf26bfcc5c98442ab5af3717927f7038', 'request_id': 'req-2bf01ea0-2b2a-4785-813a-7ac99307f602'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.256563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf26bfcc5c98442ab5af3717927f7038', 'request_id': 'req-2bf01ea0-2b2a-4785-813a-7ac99307f602'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.268741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf26bfcc5c98442ab5af3717927f7038', 'request_id': 'req-2bf01ea0-2b2a-4785-813a-7ac99307f602'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.297339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] The token cache is being invalidate because project bf26bfcc5c98442ab5af3717927f7038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:15.297674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.298448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.302380 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1067/2155] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:15 2022] DELETE /identity/v3/projects/bf26bfcc5c98442ab5af3717927f7038 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:15.306138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244e729-aa9f-4214-b1d8-e523190bd1d4 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:15.319531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244e729-aa9f-4214-b1d8-e523190bd1d4 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:15.436483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:15.437370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:15.437881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:15.438219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] PATH_INFO: `/v3/users/bc56793fe99c45afb8fc23fd5f2c342f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:15.440132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc56793fe99c45afb8fc23fd5f2c342f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:15.442988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:15.501266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:15.503547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:15.528905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf01ea0-2b2a-4785-813a-7ac99307f602 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:15.537116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc56793fe99c45afb8fc23fd5f2c342f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.542100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:15.542989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:15.543441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:15.543863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] PATH_INFO: `/v3/projects/054907db3740425db8d593526f212875` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:15.545176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=054907db3740425db8d593526f212875)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:15.546836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc56793fe99c45afb8fc23fd5f2c342f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.547560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc56793fe99c45afb8fc23fd5f2c342f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.548174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc56793fe99c45afb8fc23fd5f2c342f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.548760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:15.555579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:15.566344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc56793fe99c45afb8fc23fd5f2c342f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.645242 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1089/2156] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:15 2022] DELETE /identity/v3/users/bc56793fe99c45afb8fc23fd5f2c342f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:15.660225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '054907db3740425db8d593526f212875', 'request_id': 'req-2b1fdadd-8586-46cd-9493-a288d2f01711'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.680757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '054907db3740425db8d593526f212875', 'request_id': 'req-2b1fdadd-8586-46cd-9493-a288d2f01711'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.691124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '054907db3740425db8d593526f212875', 'request_id': 'req-2b1fdadd-8586-46cd-9493-a288d2f01711'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.729561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] The token cache is being invalidate because project 054907db3740425db8d593526f212875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:15.730311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.731170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:15.734576 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1068/2157] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:15 2022] DELETE /identity/v3/projects/054907db3740425db8d593526f212875 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:15.745871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:15.747810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:15.912088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ca4f8-5ed3-4df2-85b9-fc46983edd0d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:15.938636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:15.939640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:15.940025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:15.940405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] PATH_INFO: `/v3/users/a1f971f041d54ef684521814774fcd19` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:15.944263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f971f041d54ef684521814774fcd19)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:15.946668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:16.025766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f971f041d54ef684521814774fcd19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.035138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f971f041d54ef684521814774fcd19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.037510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f971f041d54ef684521814774fcd19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.038251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f971f041d54ef684521814774fcd19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.054075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b66dacc8-acf6-49fc-96b5-723d181f070d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f971f041d54ef684521814774fcd19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.079985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:16.081777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:16.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1090/2158] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:15 2022] DELETE /identity/v3/users/a1f971f041d54ef684521814774fcd19 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:16.172657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:16.235411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fdadd-8586-46cd-9493-a288d2f01711 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:16.416938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:16.417887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:16.418303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:16.419252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] PATH_INFO: `/v3/projects/e2fa947d1aac49f4b3dde35d21b56370` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:16.420765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2fa947d1aac49f4b3dde35d21b56370)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:16.425041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:16.525146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2fa947d1aac49f4b3dde35d21b56370', 'request_id': 'req-d096d53d-cff4-418e-9cc0-9187d86ca987'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.538123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2fa947d1aac49f4b3dde35d21b56370', 'request_id': 'req-d096d53d-cff4-418e-9cc0-9187d86ca987'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.559834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2fa947d1aac49f4b3dde35d21b56370', 'request_id': 'req-d096d53d-cff4-418e-9cc0-9187d86ca987'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.608668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] The token cache is being invalidate because project e2fa947d1aac49f4b3dde35d21b56370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:16.609215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.610093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.611691 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1069/2159] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:16 2022] DELETE /identity/v3/projects/e2fa947d1aac49f4b3dde35d21b56370 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:16.617348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f40b11-330c-4da9-90ec-9ac1712f5cab None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:16.617935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f40b11-330c-4da9-90ec-9ac1712f5cab None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:16.618038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f40b11-330c-4da9-90ec-9ac1712f5cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:16.648888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:16.651621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:16.696892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b5f40b11-330c-4da9-90ec-9ac1712f5cab None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:16.698086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b5f40b11-330c-4da9-90ec-9ac1712f5cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:16.835585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096d53d-cff4-418e-9cc0-9187d86ca987 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:16.838375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:16.843080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:16.843955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:16.844418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:16.844796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] PATH_INFO: `/v3/projects/0005547a8f284488a8876ef1614cd0e2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:16.846213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0005547a8f284488a8876ef1614cd0e2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:16.851862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:16.865709 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1091/2160] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:16.872025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c03f7-025c-4f18-86db-16aae0eca8a7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:16.875959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c03f7-025c-4f18-86db-16aae0eca8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:16.876597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c03f7-025c-4f18-86db-16aae0eca8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:16.885828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:16.943695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-124c03f7-025c-4f18-86db-16aae0eca8a7 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:16.947265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-124c03f7-025c-4f18-86db-16aae0eca8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:16.963554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0005547a8f284488a8876ef1614cd0e2', 'request_id': 'req-75c8bd33-4d6a-46f6-a565-352f4a8037eb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.983244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0005547a8f284488a8876ef1614cd0e2', 'request_id': 'req-75c8bd33-4d6a-46f6-a565-352f4a8037eb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:16.995347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0005547a8f284488a8876ef1614cd0e2', 'request_id': 'req-75c8bd33-4d6a-46f6-a565-352f4a8037eb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:17.024701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] The token cache is being invalidate because project 0005547a8f284488a8876ef1614cd0e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:17.027200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:17.029589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-75c8bd33-4d6a-46f6-a565-352f4a8037eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:17.032681 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1070/2161] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:16 2022] DELETE /identity/v3/projects/0005547a8f284488a8876ef1614cd0e2 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:17.038623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa5a4-c045-4ca6-af70-17a12f840a17 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.039243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa5a4-c045-4ca6-af70-17a12f840a17 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.040606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa5a4-c045-4ca6-af70-17a12f840a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.060969 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1092/2162] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:17.066136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c03f7-025c-4f18-86db-16aae0eca8a7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.081307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c03f7-025c-4f18-86db-16aae0eca8a7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.089939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2b818-4930-4ad5-af66-98cecbe531fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.092427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2b818-4930-4ad5-af66-98cecbe531fb admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.092716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2b818-4930-4ad5-af66-98cecbe531fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.093773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2b818-4930-4ad5-af66-98cecbe531fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.095659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9dafa5a4-c045-4ca6-af70-17a12f840a17 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:17.096368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2b818-4930-4ad5-af66-98cecbe531fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:17.096906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2b818-4930-4ad5-af66-98cecbe531fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.098697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9dafa5a4-c045-4ca6-af70-17a12f840a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:17.109469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2b818-4930-4ad5-af66-98cecbe531fb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.131851 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1093/2163] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:17.133296 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1071/2164] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:17.135793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c875b48-7dbd-431e-b3cf-4f548293e814 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.136218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c875b48-7dbd-431e-b3cf-4f548293e814 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.137131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafa5a4-c045-4ca6-af70-17a12f840a17 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.137274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c875b48-7dbd-431e-b3cf-4f548293e814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.168160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafa5a4-c045-4ca6-af70-17a12f840a17 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.175704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a071-0a65-4b0b-846b-5f34772d41b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.183907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a071-0a65-4b0b-846b-5f34772d41b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.184217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a071-0a65-4b0b-846b-5f34772d41b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.185824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1a071-0a65-4b0b-846b-5f34772d41b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.187889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a071-0a65-4b0b-846b-5f34772d41b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.191038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0c875b48-7dbd-431e-b3cf-4f548293e814 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:17.193480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c875b48-7dbd-431e-b3cf-4f548293e814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:17.212640 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1094/2165] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:17.220696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1a071-0a65-4b0b-846b-5f34772d41b9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.222266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c875b48-7dbd-431e-b3cf-4f548293e814 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.247542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c875b48-7dbd-431e-b3cf-4f548293e814 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.258169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.258810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.266902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.267377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.268283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:17.268767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.275513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.292567 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1072/2166] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:17 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:17.297967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a071-0a65-4b0b-846b-5f34772d41b9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.331731 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1095/2167] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:17.356236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.369364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c234-ae19-4509-954d-d4f8813bbecc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.444086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1a071-0a65-4b0b-846b-5f34772d41b9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.449275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca5a48-d155-40fd-9840-164bbb22bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.450064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca5a48-d155-40fd-9840-164bbb22bae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.450441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca5a48-d155-40fd-9840-164bbb22bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.451122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca5a48-d155-40fd-9840-164bbb22bae1 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.452149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca5a48-d155-40fd-9840-164bbb22bae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.455036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca5a48-d155-40fd-9840-164bbb22bae1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.504038 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1073/2168] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:17 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:17.508626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca5a48-d155-40fd-9840-164bbb22bae1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.524590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca5a48-d155-40fd-9840-164bbb22bae1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.531138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d3fe2-a40e-40ed-a2ce-83288cc30e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.534171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d3fe2-a40e-40ed-a2ce-83288cc30e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.534677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d3fe2-a40e-40ed-a2ce-83288cc30e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.535167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d3fe2-a40e-40ed-a2ce-83288cc30e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.536655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d3fe2-a40e-40ed-a2ce-83288cc30e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.539451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d3fe2-a40e-40ed-a2ce-83288cc30e8b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.548708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f30c4-d3fa-4b38-8cbd-673d70cfc445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.550340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f30c4-d3fa-4b38-8cbd-673d70cfc445 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.550873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f30c4-d3fa-4b38-8cbd-673d70cfc445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.551548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f30c4-d3fa-4b38-8cbd-673d70cfc445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.552741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f30c4-d3fa-4b38-8cbd-673d70cfc445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.555348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f30c4-d3fa-4b38-8cbd-673d70cfc445 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.574155 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1074/2169] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:17.579470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d3fe2-a40e-40ed-a2ce-83288cc30e8b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.594393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d3fe2-a40e-40ed-a2ce-83288cc30e8b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.602600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a17e2-05fa-42b4-9a63-a48d0029eb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.603766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a17e2-05fa-42b4-9a63-a48d0029eb61 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.604186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a17e2-05fa-42b4-9a63-a48d0029eb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.604563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a17e2-05fa-42b4-9a63-a48d0029eb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.605544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a17e2-05fa-42b4-9a63-a48d0029eb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.607327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a17e2-05fa-42b4-9a63-a48d0029eb61 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.613581 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1096/2170] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:17 2022] POST /identity/v3/projects => generated 383 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:17.617540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f30c4-d3fa-4b38-8cbd-673d70cfc445 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.626093 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1075/2171] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:17.627936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a17e2-05fa-42b4-9a63-a48d0029eb61 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.671661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a17e2-05fa-42b4-9a63-a48d0029eb61 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.714955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:17.733512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:17.739812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f30c4-d3fa-4b38-8cbd-673d70cfc445 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.745735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07c156-82aa-4e31-b4f4-0719edf8e506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.746907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07c156-82aa-4e31-b4f4-0719edf8e506 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.747103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07c156-82aa-4e31-b4f4-0719edf8e506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.747524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07c156-82aa-4e31-b4f4-0719edf8e506 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.748524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07c156-82aa-4e31-b4f4-0719edf8e506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.751039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07c156-82aa-4e31-b4f4-0719edf8e506 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.756632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cc4e6-8ca4-4136-abe0-242039285513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.757578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc4e6-8ca4-4136-abe0-242039285513 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.758073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc4e6-8ca4-4136-abe0-242039285513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.758604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc4e6-8ca4-4136-abe0-242039285513 admin admin] PATH_INFO: `/v3/projects/b139144c40074965b405ddd5acdfbbd3/users/1aa98cdbbe2a49de8a7d133ed8f746d6/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.780954 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1097/2172] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:17 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:17.782141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07c156-82aa-4e31-b4f4-0719edf8e506 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.784677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc4e6-8ca4-4136-abe0-242039285513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b139144c40074965b405ddd5acdfbbd3, user_id=1aa98cdbbe2a49de8a7d133ed8f746d6, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.791455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc4e6-8ca4-4136-abe0-242039285513 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.793602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07c156-82aa-4e31-b4f4-0719edf8e506 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.799991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09012a06-81cf-4f95-926c-0b9404f8f89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.800717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09012a06-81cf-4f95-926c-0b9404f8f89c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.801831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09012a06-81cf-4f95-926c-0b9404f8f89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.802166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09012a06-81cf-4f95-926c-0b9404f8f89c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.802871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09012a06-81cf-4f95-926c-0b9404f8f89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.804528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09012a06-81cf-4f95-926c-0b9404f8f89c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.817680 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1098/2174] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:17.821112 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1076/2174] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:17 2022] PUT /identity/v3/projects/b139144c40074965b405ddd5acdfbbd3/users/1aa98cdbbe2a49de8a7d133ed8f746d6/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:17.822008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09012a06-81cf-4f95-926c-0b9404f8f89c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.822691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f54391-50fe-4fb0-87bd-f48b11eb7fb1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.823210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f54391-50fe-4fb0-87bd-f48b11eb7fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.823391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f54391-50fe-4fb0-87bd-f48b11eb7fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.832798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09012a06-81cf-4f95-926c-0b9404f8f89c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.864375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-57f54391-50fe-4fb0-87bd-f48b11eb7fb1 None None] MFA Rules not processed for user `1aa98cdbbe2a49de8a7d133ed8f746d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:17.866129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-57f54391-50fe-4fb0-87bd-f48b11eb7fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:17.908750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c695e-d48e-45f4-bda8-c7cda70dd1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.909364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c695e-d48e-45f4-bda8-c7cda70dd1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.909657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c695e-d48e-45f4-bda8-c7cda70dd1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.909953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c695e-d48e-45f4-bda8-c7cda70dd1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.911173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c695e-d48e-45f4-bda8-c7cda70dd1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.913224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c695e-d48e-45f4-bda8-c7cda70dd1c9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.929510 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1099/2175] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:17.931025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c695e-d48e-45f4-bda8-c7cda70dd1c9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.941175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c695e-d48e-45f4-bda8-c7cda70dd1c9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:17.947862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fde05-45c6-494f-9ef3-cb24facab8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:17.948606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fde05-45c6-494f-9ef3-cb24facab8c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:17.948969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fde05-45c6-494f-9ef3-cb24facab8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:17.949332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fde05-45c6-494f-9ef3-cb24facab8c2 admin admin] PATH_INFO: `/v3/projects/b6fd0b31923f41578bcf28d284d34eeb/users/de237d9a5af340f086243e66029a9068/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:17.969008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fde05-45c6-494f-9ef3-cb24facab8c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fd0b31923f41578bcf28d284d34eeb, user_id=de237d9a5af340f086243e66029a9068, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:17.971708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fde05-45c6-494f-9ef3-cb24facab8c2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:17.982090 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1077/2176] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:17.996464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f54391-50fe-4fb0-87bd-f48b11eb7fb1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:17.998110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-57f54391-50fe-4fb0-87bd-f48b11eb7fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:18.006344 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1100/2177] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:17 2022] PUT /identity/v3/projects/b6fd0b31923f41578bcf28d284d34eeb/users/de237d9a5af340f086243e66029a9068/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:18.021752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887bbb2-8577-46f6-bc42-1ca6a6c07403 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.022186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887bbb2-8577-46f6-bc42-1ca6a6c07403 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.022393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887bbb2-8577-46f6-bc42-1ca6a6c07403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:18.059005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1887bbb2-8577-46f6-bc42-1ca6a6c07403 None None] MFA Rules not processed for user `de237d9a5af340f086243e66029a9068`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:18.060252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1887bbb2-8577-46f6-bc42-1ca6a6c07403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:18.166270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f54391-50fe-4fb0-87bd-f48b11eb7fb1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:18.169590 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1101/2178] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:18.171770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887bbb2-8577-46f6-bc42-1ca6a6c07403 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:18.222813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:18.266281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a7f05-610c-4b66-af4d-6f6b10f8b97f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:18.267073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a7f05-610c-4b66-af4d-6f6b10f8b97f service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.267446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a7f05-610c-4b66-af4d-6f6b10f8b97f service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.267815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a7f05-610c-4b66-af4d-6f6b10f8b97f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:18.277276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887bbb2-8577-46f6-bc42-1ca6a6c07403 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:18.286699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb47d0-34e7-4852-820f-150fd5a848a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:18.287523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb47d0-34e7-4852-820f-150fd5a848a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.287825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb47d0-34e7-4852-820f-150fd5a848a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.289309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb47d0-34e7-4852-820f-150fd5a848a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:18.290215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb47d0-34e7-4852-820f-150fd5a848a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:18.292571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb47d0-34e7-4852-820f-150fd5a848a9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:18.321183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:18.340874 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1102/2179] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:18 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:18.345180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb47d0-34e7-4852-820f-150fd5a848a9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:18.365716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a7f05-610c-4b66-af4d-6f6b10f8b97f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:18.367230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a7f05-610c-4b66-af4d-6f6b10f8b97f service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:18.382881 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1078/2180] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:18.490296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb47d0-34e7-4852-820f-150fd5a848a9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:18.504836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c52835-fad4-43d0-9924-33bcc9b89615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:18.507698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c52835-fad4-43d0-9924-33bcc9b89615 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.508534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c52835-fad4-43d0-9924-33bcc9b89615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.509137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c52835-fad4-43d0-9924-33bcc9b89615 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:18.514425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c52835-fad4-43d0-9924-33bcc9b89615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:18.518896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c52835-fad4-43d0-9924-33bcc9b89615 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:18.547678 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1103/2181] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:18 2022] POST /identity/v3/users => generated 298 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:18.552550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a7f05-610c-4b66-af4d-6f6b10f8b97f service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:18.571269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a7f05-610c-4b66-af4d-6f6b10f8b97f service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:18.578016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecd3e6-dc09-4b6a-b2de-3ebcf5296855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:18.579094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecd3e6-dc09-4b6a-b2de-3ebcf5296855 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.579601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecd3e6-dc09-4b6a-b2de-3ebcf5296855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.580069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecd3e6-dc09-4b6a-b2de-3ebcf5296855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:18.581151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecd3e6-dc09-4b6a-b2de-3ebcf5296855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:18.584332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecd3e6-dc09-4b6a-b2de-3ebcf5296855 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:18.599003 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1079/2182] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:18.604618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c52835-fad4-43d0-9924-33bcc9b89615 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:18.622662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c52835-fad4-43d0-9924-33bcc9b89615 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:18.631606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549266be-20a3-4224-8a6f-ef4158cb60b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:18.632367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549266be-20a3-4224-8a6f-ef4158cb60b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.632627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549266be-20a3-4224-8a6f-ef4158cb60b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.632912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549266be-20a3-4224-8a6f-ef4158cb60b5 admin admin] PATH_INFO: `/v3/projects/71903efa7590495390dd5c65fa42b019/users/bcbc078eb36b4c47a62d96424f768054/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:18.665154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549266be-20a3-4224-8a6f-ef4158cb60b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71903efa7590495390dd5c65fa42b019, user_id=bcbc078eb36b4c47a62d96424f768054, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:18.668244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549266be-20a3-4224-8a6f-ef4158cb60b5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:18.704816 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1104/2183] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:18 2022] PUT /identity/v3/projects/71903efa7590495390dd5c65fa42b019/users/bcbc078eb36b4c47a62d96424f768054/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:18.710961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecd3e6-dc09-4b6a-b2de-3ebcf5296855 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:18.728002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecd3e6-dc09-4b6a-b2de-3ebcf5296855 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:18.826417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba5eac-5a10-41b8-ae9b-a962b9ea1d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:18.827972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba5eac-5a10-41b8-ae9b-a962b9ea1d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.827972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba5eac-5a10-41b8-ae9b-a962b9ea1d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.827972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba5eac-5a10-41b8-ae9b-a962b9ea1d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:18.828557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba5eac-5a10-41b8-ae9b-a962b9ea1d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:18.830783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba5eac-5a10-41b8-ae9b-a962b9ea1d4b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:18.849896 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1080/2184] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:18.853713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549266be-20a3-4224-8a6f-ef4158cb60b5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:18.874725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549266be-20a3-4224-8a6f-ef4158cb60b5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:18.884857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a9c60-d6d6-4ddb-9f8d-59950cd2fd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:18.886394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9c60-d6d6-4ddb-9f8d-59950cd2fd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.886760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9c60-d6d6-4ddb-9f8d-59950cd2fd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.887791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a9c60-d6d6-4ddb-9f8d-59950cd2fd8b admin admin] PATH_INFO: `/v3/projects/71903efa7590495390dd5c65fa42b019/users/bcbc078eb36b4c47a62d96424f768054/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:18.892235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a9c60-d6d6-4ddb-9f8d-59950cd2fd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=71903efa7590495390dd5c65fa42b019, user_id=bcbc078eb36b4c47a62d96424f768054, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:18.910144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a9c60-d6d6-4ddb-9f8d-59950cd2fd8b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:18.933212 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1105/2185] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:18 2022] PUT /identity/v3/projects/71903efa7590495390dd5c65fa42b019/users/bcbc078eb36b4c47a62d96424f768054/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:18.948785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce14ddd-636e-49f1-844a-7bc50774ab56 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:18.949125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce14ddd-636e-49f1-844a-7bc50774ab56 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:18.949417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce14ddd-636e-49f1-844a-7bc50774ab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:19.003260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8ce14ddd-636e-49f1-844a-7bc50774ab56 None None] MFA Rules not processed for user `bcbc078eb36b4c47a62d96424f768054`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:19.006413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8ce14ddd-636e-49f1-844a-7bc50774ab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:19.141390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a9c60-d6d6-4ddb-9f8d-59950cd2fd8b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:19.188990 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1081/2186] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:19.208122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce14ddd-636e-49f1-844a-7bc50774ab56 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:19.211329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8ce14ddd-636e-49f1-844a-7bc50774ab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:19.242074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a9c60-d6d6-4ddb-9f8d-59950cd2fd8b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:19.246099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:19.246721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:19.247064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:19.247274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:19.249607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:19.269695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:19.280183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:19.295914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce14ddd-636e-49f1-844a-7bc50774ab56 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:19.301071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09248190-3a70-41c7-b38d-ce17395136c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:19.302526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09248190-3a70-41c7-b38d-ce17395136c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:19.302949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09248190-3a70-41c7-b38d-ce17395136c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:19.303331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09248190-3a70-41c7-b38d-ce17395136c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:19.314685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09248190-3a70-41c7-b38d-ce17395136c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:19.317357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09248190-3a70-41c7-b38d-ce17395136c3 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:19.329037 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1082/2187] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:19.335503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:19.337204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:19.351061 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1106/2188] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:19.443764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09248190-3a70-41c7-b38d-ce17395136c3 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:19.456153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09248190-3a70-41c7-b38d-ce17395136c3 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:19.466851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f62d7-4bc1-4b87-aa8b-b2bf0ef020bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:19.467598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f62d7-4bc1-4b87-aa8b-b2bf0ef020bb service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:19.467749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f62d7-4bc1-4b87-aa8b-b2bf0ef020bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:19.468007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f62d7-4bc1-4b87-aa8b-b2bf0ef020bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:19.610519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f62d7-4bc1-4b87-aa8b-b2bf0ef020bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:19.611988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f62d7-4bc1-4b87-aa8b-b2bf0ef020bb service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:19.641739 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1083/2189] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:20.057517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:20.163498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5102-09d6-44b7-be36-011fb8de0730 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:20.168603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:20.170261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:20.170760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:20.171213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] PATH_INFO: `/v3/users/de237d9a5af340f086243e66029a9068` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:20.172748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de237d9a5af340f086243e66029a9068)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:20.175585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:20.275492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de237d9a5af340f086243e66029a9068'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:20.286187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de237d9a5af340f086243e66029a9068'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:20.286933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de237d9a5af340f086243e66029a9068'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:20.287694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de237d9a5af340f086243e66029a9068'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:20.306082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de237d9a5af340f086243e66029a9068'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:20.337830 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1107/2190] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:20 2022] DELETE /identity/v3/users/de237d9a5af340f086243e66029a9068 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:20.342281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f0d24-b561-40fd-8473-830cd7368bad None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:20.342752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f0d24-b561-40fd-8473-830cd7368bad None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:20.342829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f0d24-b561-40fd-8473-830cd7368bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:20.366895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d47f0d24-b561-40fd-8473-830cd7368bad None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:20.368340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d47f0d24-b561-40fd-8473-830cd7368bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:20.497540 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1084/2191] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:20.504866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:20.507152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:20.518632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcd69-dc54-47ab-afad-a02c3ee0b295 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:20.519105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcd69-dc54-47ab-afad-a02c3ee0b295 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:20.519511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bcd69-dc54-47ab-afad-a02c3ee0b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:20.575206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4d0bcd69-dc54-47ab-afad-a02c3ee0b295 None None] MFA Rules not processed for user `7e04471e8c634a6586789f82204241a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:20.576846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4d0bcd69-dc54-47ab-afad-a02c3ee0b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:20.599936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:20.642074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafba8d-43b1-4e89-9265-04ae1202c548 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:20.654269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d6b10-8678-4b7d-a6ba-99803c7771b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:20.654913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d6b10-8678-4b7d-a6ba-99803c7771b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:20.655478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d6b10-8678-4b7d-a6ba-99803c7771b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:20.655864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d6b10-8678-4b7d-a6ba-99803c7771b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:20.681885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d6b10-8678-4b7d-a6ba-99803c7771b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:20.681885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d6b10-8678-4b7d-a6ba-99803c7771b8 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:20.695972 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1108/2192] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:20.746394 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1085/2193] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:20.747358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53222d55-1bfb-4fd1-910b-a46c33f29e77 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:20.747497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53222d55-1bfb-4fd1-910b-a46c33f29e77 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:20.747788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53222d55-1bfb-4fd1-910b-a46c33f29e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:20.783969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-53222d55-1bfb-4fd1-910b-a46c33f29e77 None None] MFA Rules not processed for user `7e04471e8c634a6586789f82204241a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:20.785728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-53222d55-1bfb-4fd1-910b-a46c33f29e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:20.806761 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1109/2194] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:20.822104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bcd69-dc54-47ab-afad-a02c3ee0b295 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:20.823707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4d0bcd69-dc54-47ab-afad-a02c3ee0b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:20.846101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:20.949448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bcd69-dc54-47ab-afad-a02c3ee0b295 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:20.956432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e7dd9-cfad-4290-942f-0f4625c16af1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:20.957681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7dd9-cfad-4290-942f-0f4625c16af1 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:20.957978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7dd9-cfad-4290-942f-0f4625c16af1 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:20.958453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e7dd9-cfad-4290-942f-0f4625c16af1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:20.969099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e7dd9-cfad-4290-942f-0f4625c16af1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:20.972282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e7dd9-cfad-4290-942f-0f4625c16af1 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:20.991227 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1086/2195] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:21.154743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53222d55-1bfb-4fd1-910b-a46c33f29e77 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:21.156730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-53222d55-1bfb-4fd1-910b-a46c33f29e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:21.184976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53222d55-1bfb-4fd1-910b-a46c33f29e77 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:21.191690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:21.192818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:21.193258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:21.193707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] PATH_INFO: `/v3/projects/b6fd0b31923f41578bcf28d284d34eeb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:21.195068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fd0b31923f41578bcf28d284d34eeb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:21.198580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:21.280158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fd0b31923f41578bcf28d284d34eeb', 'request_id': 'req-24a515c3-385d-464b-aa5d-77788113cf39'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.294525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fd0b31923f41578bcf28d284d34eeb', 'request_id': 'req-24a515c3-385d-464b-aa5d-77788113cf39'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.312374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fd0b31923f41578bcf28d284d34eeb', 'request_id': 'req-24a515c3-385d-464b-aa5d-77788113cf39'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.336730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] The token cache is being invalidate because project b6fd0b31923f41578bcf28d284d34eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:21.337754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.338440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.340587 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1110/2196] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:21 2022] DELETE /identity/v3/projects/b6fd0b31923f41578bcf28d284d34eeb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:21.344808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e7dd9-cfad-4290-942f-0f4625c16af1 service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:21.361001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e7dd9-cfad-4290-942f-0f4625c16af1 service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:21.454412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:21.455085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:21.455368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:21.455668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] PATH_INFO: `/v3/users/bcbc078eb36b4c47a62d96424f768054` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:21.458716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcbc078eb36b4c47a62d96424f768054)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:21.460946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:21.536599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcbc078eb36b4c47a62d96424f768054'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.545137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcbc078eb36b4c47a62d96424f768054'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.545991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcbc078eb36b4c47a62d96424f768054'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.546684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcbc078eb36b4c47a62d96424f768054'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.557849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3402b526-98e8-4001-a1a7-f833d16a1fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcbc078eb36b4c47a62d96424f768054'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:21.578891 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1087/2197] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:21 2022] DELETE /identity/v3/users/bcbc078eb36b4c47a62d96424f768054 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:21.884895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:21.886803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:21.913009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a515c3-385d-464b-aa5d-77788113cf39 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:21.974617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:21.975292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:21.975618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:21.975915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] PATH_INFO: `/v3/projects/71903efa7590495390dd5c65fa42b019` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:21.977087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71903efa7590495390dd5c65fa42b019)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:21.979134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:22.033221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71903efa7590495390dd5c65fa42b019', 'request_id': 'req-617a8688-4951-45d3-bd34-83f7a0175a14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:22.043964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71903efa7590495390dd5c65fa42b019', 'request_id': 'req-617a8688-4951-45d3-bd34-83f7a0175a14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:22.046939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:22.053673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71903efa7590495390dd5c65fa42b019', 'request_id': 'req-617a8688-4951-45d3-bd34-83f7a0175a14'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:22.070259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] The token cache is being invalidate because project 71903efa7590495390dd5c65fa42b019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:22.070875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:22.071686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-617a8688-4951-45d3-bd34-83f7a0175a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:22.073701 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1111/2198] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:21 2022] DELETE /identity/v3/projects/71903efa7590495390dd5c65fa42b019 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:22.078843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721205f-3a4e-424b-81c5-078f60e82a6d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.079651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721205f-3a4e-424b-81c5-078f60e82a6d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.080115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721205f-3a4e-424b-81c5-078f60e82a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.107593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3721205f-3a4e-424b-81c5-078f60e82a6d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:22.108907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3721205f-3a4e-424b-81c5-078f60e82a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:22.121762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:22.183209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18831e63-dbcf-4586-b68f-68fb99894970 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.184681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18831e63-dbcf-4586-b68f-68fb99894970 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.185186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18831e63-dbcf-4586-b68f-68fb99894970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.216455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-18831e63-dbcf-4586-b68f-68fb99894970 None None] MFA Rules not processed for user `7e04471e8c634a6586789f82204241a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:22.218359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-18831e63-dbcf-4586-b68f-68fb99894970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:22.243584 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1088/2199] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:22.247262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c327ef-51c8-4dce-a096-c1a3a001e444 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.247547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c327ef-51c8-4dce-a096-c1a3a001e444 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.247828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c327ef-51c8-4dce-a096-c1a3a001e444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.281390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a6c327ef-51c8-4dce-a096-c1a3a001e444 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:22.282653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a6c327ef-51c8-4dce-a096-c1a3a001e444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:22.292519 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1089/2200] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:22.297579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c327ef-51c8-4dce-a096-c1a3a001e444 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:22.312147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c327ef-51c8-4dce-a096-c1a3a001e444 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:22.318439 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1112/2201] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:22.319378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:22.320003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.320349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.320621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.321317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:22.321644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:22.341161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:22.361897 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1090/2202] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:22.367004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18831e63-dbcf-4586-b68f-68fb99894970 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:22.387143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18831e63-dbcf-4586-b68f-68fb99894970 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:22.395654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f522-ef35-44d6-a424-eb6de0285dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:22.397305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f522-ef35-44d6-a424-eb6de0285dec admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.398399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f522-ef35-44d6-a424-eb6de0285dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.399266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819f522-ef35-44d6-a424-eb6de0285dec admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.400023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819f522-ef35-44d6-a424-eb6de0285dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:22.402410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819f522-ef35-44d6-a424-eb6de0285dec admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:22.452616 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1113/2203] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:22 2022] POST /identity/v3/projects => generated 377 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:22.456702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:22.476743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f454c4-a7d3-4e17-be00-00eaac732257 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:22.651496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49baee-0ecd-4774-97c0-4bafe5437f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:22.652187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49baee-0ecd-4774-97c0-4bafe5437f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.652580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49baee-0ecd-4774-97c0-4bafe5437f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.652930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49baee-0ecd-4774-97c0-4bafe5437f75 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.654450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49baee-0ecd-4774-97c0-4bafe5437f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:22.659491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49baee-0ecd-4774-97c0-4bafe5437f75 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:22.705660 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1091/2204] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:22 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:22.710222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f522-ef35-44d6-a424-eb6de0285dec admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:22.724514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819f522-ef35-44d6-a424-eb6de0285dec admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:22.738427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e72839-abb6-4bb9-813a-4918eefadaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:22.748117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e72839-abb6-4bb9-813a-4918eefadaff admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.748589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e72839-abb6-4bb9-813a-4918eefadaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.749022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e72839-abb6-4bb9-813a-4918eefadaff admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.749833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e72839-abb6-4bb9-813a-4918eefadaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:22.751935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e72839-abb6-4bb9-813a-4918eefadaff admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:22.777885 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1114/2205] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:22.779317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49baee-0ecd-4774-97c0-4bafe5437f75 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:22.795259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49baee-0ecd-4774-97c0-4bafe5437f75 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:22.809750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836f641-6f5d-4557-bb81-8f4ccd64692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:22.811678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836f641-6f5d-4557-bb81-8f4ccd64692f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.812132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836f641-6f5d-4557-bb81-8f4ccd64692f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.812904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836f641-6f5d-4557-bb81-8f4ccd64692f admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.815469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836f641-6f5d-4557-bb81-8f4ccd64692f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:22.819116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836f641-6f5d-4557-bb81-8f4ccd64692f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:22.847572 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1092/2206] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:22.864145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e72839-abb6-4bb9-813a-4918eefadaff admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:22.876602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e72839-abb6-4bb9-813a-4918eefadaff admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:22.891145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b1806-ad98-4374-acba-ad00e3a43c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:22.891892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1806-ad98-4374-acba-ad00e3a43c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.892275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1806-ad98-4374-acba-ad00e3a43c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.892624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1806-ad98-4374-acba-ad00e3a43c21 admin admin] PATH_INFO: `/v3/projects/4a0e8c1869b647449387361c894de25a/users/ffd4fd8a8ad045e099c7be28a814c7d4/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:22.932188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b1806-ad98-4374-acba-ad00e3a43c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0e8c1869b647449387361c894de25a, user_id=ffd4fd8a8ad045e099c7be28a814c7d4, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:22.937515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b1806-ad98-4374-acba-ad00e3a43c21 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:22.970088 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1115/2207] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:22 2022] PUT /identity/v3/projects/4a0e8c1869b647449387361c894de25a/users/ffd4fd8a8ad045e099c7be28a814c7d4/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:22.987443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d2f1-613a-4b7b-aaf9-3e1d80b9423c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:22.987443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d2f1-613a-4b7b-aaf9-3e1d80b9423c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:22.987443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90d2f1-613a-4b7b-aaf9-3e1d80b9423c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:23.034802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6d90d2f1-613a-4b7b-aaf9-3e1d80b9423c None None] MFA Rules not processed for user `ffd4fd8a8ad045e099c7be28a814c7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:23.037596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6d90d2f1-613a-4b7b-aaf9-3e1d80b9423c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:23.152394 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1093/2208] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:23.153855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b1806-ad98-4374-acba-ad00e3a43c21 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:23.274269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b1806-ad98-4374-acba-ad00e3a43c21 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:23.280890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c6fd9-44a7-4670-a323-166d4948b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:23.281830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c6fd9-44a7-4670-a323-166d4948b18a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:23.282285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c6fd9-44a7-4670-a323-166d4948b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:23.282706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c6fd9-44a7-4670-a323-166d4948b18a admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:23.283521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c6fd9-44a7-4670-a323-166d4948b18a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:23.286571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c6fd9-44a7-4670-a323-166d4948b18a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:23.329324 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1116/2209] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:23 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:23.333438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90d2f1-613a-4b7b-aaf9-3e1d80b9423c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:23.352068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:23.360084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90d2f1-613a-4b7b-aaf9-3e1d80b9423c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:23.557110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238ba28-1072-46b5-81cd-5214ba2ea9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:23.558419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ba28-1072-46b5-81cd-5214ba2ea9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:23.558858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ba28-1072-46b5-81cd-5214ba2ea9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:23.559290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238ba28-1072-46b5-81cd-5214ba2ea9d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:23.560085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238ba28-1072-46b5-81cd-5214ba2ea9d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:23.562171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238ba28-1072-46b5-81cd-5214ba2ea9d3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:23.597737 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1094/2210] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:23 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:23.601901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c6fd9-44a7-4670-a323-166d4948b18a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:23.618300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c6fd9-44a7-4670-a323-166d4948b18a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:23.631510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a45df9-644a-4886-b1be-58f47b6776d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:23.633899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a45df9-644a-4886-b1be-58f47b6776d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:23.634333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a45df9-644a-4886-b1be-58f47b6776d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:23.635967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a45df9-644a-4886-b1be-58f47b6776d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:23.636759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a45df9-644a-4886-b1be-58f47b6776d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:23.641417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a45df9-644a-4886-b1be-58f47b6776d3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:23.669501 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1117/2211] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:23.672173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238ba28-1072-46b5-81cd-5214ba2ea9d3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:23.688922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238ba28-1072-46b5-81cd-5214ba2ea9d3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:23.701558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d604fb6-8c33-46a8-a4bb-d7474ae6217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:23.703228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604fb6-8c33-46a8-a4bb-d7474ae6217d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:23.705128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604fb6-8c33-46a8-a4bb-d7474ae6217d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:23.705345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604fb6-8c33-46a8-a4bb-d7474ae6217d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:23.706151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d604fb6-8c33-46a8-a4bb-d7474ae6217d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:23.727253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d604fb6-8c33-46a8-a4bb-d7474ae6217d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:23.773044 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1095/2212] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:23.773641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a45df9-644a-4886-b1be-58f47b6776d3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:23.797253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a45df9-644a-4886-b1be-58f47b6776d3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:23.812515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0138202-1877-4c2b-9acf-e7716a3d59d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:23.813361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0138202-1877-4c2b-9acf-e7716a3d59d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:23.814174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0138202-1877-4c2b-9acf-e7716a3d59d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:23.816046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0138202-1877-4c2b-9acf-e7716a3d59d6 admin admin] PATH_INFO: `/v3/projects/da0fa7fbdb93488dbb1c151991aa98f4/users/e349e6a72df046ad8e5e3d43e0bd7dd2/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:23.842015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:23.857060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0138202-1877-4c2b-9acf-e7716a3d59d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da0fa7fbdb93488dbb1c151991aa98f4, user_id=e349e6a72df046ad8e5e3d43e0bd7dd2, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:23.871995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0138202-1877-4c2b-9acf-e7716a3d59d6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:23.932464 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1118/2213] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:23 2022] PUT /identity/v3/projects/da0fa7fbdb93488dbb1c151991aa98f4/users/e349e6a72df046ad8e5e3d43e0bd7dd2/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:23.935395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebda73-d770-4b52-84bd-6246b50bd0bc None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:23.935690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebda73-d770-4b52-84bd-6246b50bd0bc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:23.936238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebda73-d770-4b52-84bd-6246b50bd0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:23.985711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5aebda73-d770-4b52-84bd-6246b50bd0bc None None] MFA Rules not processed for user `e349e6a72df046ad8e5e3d43e0bd7dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:23.985711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5aebda73-d770-4b52-84bd-6246b50bd0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:24.140043 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1096/2214] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:24.144656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0138202-1877-4c2b-9acf-e7716a3d59d6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:24.222157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0138202-1877-4c2b-9acf-e7716a3d59d6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:24.227240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595e592-df79-4712-9969-3ba05711e5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:24.227969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595e592-df79-4712-9969-3ba05711e5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:24.228257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595e592-df79-4712-9969-3ba05711e5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:24.228539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595e592-df79-4712-9969-3ba05711e5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:24.229225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595e592-df79-4712-9969-3ba05711e5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:24.231218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595e592-df79-4712-9969-3ba05711e5cf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:24.260824 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1119/2215] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:24 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:24.265804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebda73-d770-4b52-84bd-6246b50bd0bc None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:24.284220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebda73-d770-4b52-84bd-6246b50bd0bc None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:24.349241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3ca7d-91f5-48fd-9bc7-dbd29310e646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:24.350097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3ca7d-91f5-48fd-9bc7-dbd29310e646 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:24.350479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3ca7d-91f5-48fd-9bc7-dbd29310e646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:24.350753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3ca7d-91f5-48fd-9bc7-dbd29310e646 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:24.352159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3ca7d-91f5-48fd-9bc7-dbd29310e646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:24.356532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3ca7d-91f5-48fd-9bc7-dbd29310e646 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:24.383324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:24.391075 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1097/2216] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:24 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:24.396156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595e592-df79-4712-9969-3ba05711e5cf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:24.416093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595e592-df79-4712-9969-3ba05711e5cf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:24.430100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f56ee-6bad-40a1-b1cf-c5e80419dbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:24.433832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f56ee-6bad-40a1-b1cf-c5e80419dbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:24.434136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f56ee-6bad-40a1-b1cf-c5e80419dbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:24.434423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f56ee-6bad-40a1-b1cf-c5e80419dbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:24.435241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f56ee-6bad-40a1-b1cf-c5e80419dbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:24.436944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f56ee-6bad-40a1-b1cf-c5e80419dbd6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:24.473895 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1120/2217] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:24.474976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3ca7d-91f5-48fd-9bc7-dbd29310e646 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:24.505199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3ca7d-91f5-48fd-9bc7-dbd29310e646 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:24.510977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d21bcb-0fdd-4c64-b012-1134d66e0aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:24.511353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d21bcb-0fdd-4c64-b012-1134d66e0aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:24.511809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d21bcb-0fdd-4c64-b012-1134d66e0aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:24.511907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d21bcb-0fdd-4c64-b012-1134d66e0aee admin admin] PATH_INFO: `/v3/projects/113787f3f77d4c7d81df50abfdba6121/users/1598b9d5133849128867541fbc8097e3/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:24.540890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d21bcb-0fdd-4c64-b012-1134d66e0aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=113787f3f77d4c7d81df50abfdba6121, user_id=1598b9d5133849128867541fbc8097e3, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:24.543740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d21bcb-0fdd-4c64-b012-1134d66e0aee admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:24.559820 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1098/2218] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:24 2022] PUT /identity/v3/projects/113787f3f77d4c7d81df50abfdba6121/users/1598b9d5133849128867541fbc8097e3/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:24.564021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f56ee-6bad-40a1-b1cf-c5e80419dbd6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:24.575972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f56ee-6bad-40a1-b1cf-c5e80419dbd6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:24.682955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19b136-407f-4e84-846f-76abff2866e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:24.683651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19b136-407f-4e84-846f-76abff2866e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:24.684121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19b136-407f-4e84-846f-76abff2866e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:24.684448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19b136-407f-4e84-846f-76abff2866e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:24.685167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19b136-407f-4e84-846f-76abff2866e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:24.687035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19b136-407f-4e84-846f-76abff2866e2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:24.702640 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1121/2219] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:24.703776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d21bcb-0fdd-4c64-b012-1134d66e0aee admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:24.720254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d21bcb-0fdd-4c64-b012-1134d66e0aee admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:24.766012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:24.768358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:24.771318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:24.773424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] PATH_INFO: `/v3/projects/113787f3f77d4c7d81df50abfdba6121/users/1598b9d5133849128867541fbc8097e3/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:24.775170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=113787f3f77d4c7d81df50abfdba6121, user_id=1598b9d5133849128867541fbc8097e3, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:24.781078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:24.804695 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1099/2220] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:24 2022] PUT /identity/v3/projects/113787f3f77d4c7d81df50abfdba6121/users/1598b9d5133849128867541fbc8097e3/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:24.816848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5597-9803-42c7-9d04-a7c750cdc30d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:24.817677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5597-9803-42c7-9d04-a7c750cdc30d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:24.817677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a5597-9803-42c7-9d04-a7c750cdc30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:24.902673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1a8a5597-9803-42c7-9d04-a7c750cdc30d None None] MFA Rules not processed for user `1598b9d5133849128867541fbc8097e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:24.905003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1a8a5597-9803-42c7-9d04-a7c750cdc30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:24.984366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:25.061013 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1122/2221] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:25.107618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:25.109520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:25.217154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7463630-6ba8-4069-93cb-b85374ada5a4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:25.221230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5c6f-1332-4aef-b720-e7f283963f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:25.221893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5c6f-1332-4aef-b720-e7f283963f90 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:25.222399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5c6f-1332-4aef-b720-e7f283963f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:25.223079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5c6f-1332-4aef-b720-e7f283963f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:25.333675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e5c6f-1332-4aef-b720-e7f283963f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:25.335979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e5c6f-1332-4aef-b720-e7f283963f90 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:25.353130 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1100/2222] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:25.383753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a5597-9803-42c7-9d04-a7c750cdc30d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:25.386720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1a8a5597-9803-42c7-9d04-a7c750cdc30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:25.532602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a5597-9803-42c7-9d04-a7c750cdc30d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:25.546654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:25.547407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:25.547776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:25.548799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] PATH_INFO: `/v3/users/cd49cd7c67754f1fbdb1453478ef23a9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:25.552283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd49cd7c67754f1fbdb1453478ef23a9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:25.554991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:25.677787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd49cd7c67754f1fbdb1453478ef23a9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:25.685429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd49cd7c67754f1fbdb1453478ef23a9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:25.686266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd49cd7c67754f1fbdb1453478ef23a9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:25.686895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd49cd7c67754f1fbdb1453478ef23a9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:25.697598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd49cd7c67754f1fbdb1453478ef23a9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:25.736864 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1123/2223] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:25 2022] DELETE /identity/v3/users/cd49cd7c67754f1fbdb1453478ef23a9 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:25.742078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979cdd0-f313-4519-850b-0cba002f3204 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:25.742722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979cdd0-f313-4519-850b-0cba002f3204 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:25.742722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979cdd0-f313-4519-850b-0cba002f3204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:25.770066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0979cdd0-f313-4519-850b-0cba002f3204 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:25.772374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0979cdd0-f313-4519-850b-0cba002f3204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:25.923119 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1101/2224] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:25.935074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:25.937094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:26.046419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b248058-db07-4162-803a-615e321df5f9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:26.050792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1364fd-c988-4d31-9849-46b048dd8908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:26.051008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1364fd-c988-4d31-9849-46b048dd8908 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:26.051274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1364fd-c988-4d31-9849-46b048dd8908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:26.051545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1364fd-c988-4d31-9849-46b048dd8908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:26.063887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1364fd-c988-4d31-9849-46b048dd8908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:26.065463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1364fd-c988-4d31-9849-46b048dd8908 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:26.084994 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1124/2225] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:26.239999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979cdd0-f313-4519-850b-0cba002f3204 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:26.241711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0979cdd0-f313-4519-850b-0cba002f3204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:26.255914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979cdd0-f313-4519-850b-0cba002f3204 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:26.261005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:26.261913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:26.262359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:26.262774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] PATH_INFO: `/v3/users/1aa98cdbbe2a49de8a7d133ed8f746d6` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:26.264073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa98cdbbe2a49de8a7d133ed8f746d6)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:26.266222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:26.291321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:26.356448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa98cdbbe2a49de8a7d133ed8f746d6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.371541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa98cdbbe2a49de8a7d133ed8f746d6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.373507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa98cdbbe2a49de8a7d133ed8f746d6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.374650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa98cdbbe2a49de8a7d133ed8f746d6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.389260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa98cdbbe2a49de8a7d133ed8f746d6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.435181 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1102/2226] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:26 2022] DELETE /identity/v3/users/1aa98cdbbe2a49de8a7d133ed8f746d6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:26.437801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84ede6-0339-4c27-aa20-8f2473bb0d90 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:26.438230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84ede6-0339-4c27-aa20-8f2473bb0d90 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:26.438762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84ede6-0339-4c27-aa20-8f2473bb0d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:26.449460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:26.469262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9e84ede6-0339-4c27-aa20-8f2473bb0d90 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:26.470740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9e84ede6-0339-4c27-aa20-8f2473bb0d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:26.609843 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1125/2227] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:26.625412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:26.630180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:26.701687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84ede6-0339-4c27-aa20-8f2473bb0d90 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:26.703390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9e84ede6-0339-4c27-aa20-8f2473bb0d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:26.741257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84ede6-0339-4c27-aa20-8f2473bb0d90 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:26.755236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:26.756384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:26.756976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:26.757474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] PATH_INFO: `/v3/projects/9421f2a8299540af9c5dfca2965c578f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:26.758884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9421f2a8299540af9c5dfca2965c578f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:26.769214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:26.897806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9421f2a8299540af9c5dfca2965c578f', 'request_id': 'req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.910325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9421f2a8299540af9c5dfca2965c578f', 'request_id': 'req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.919997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd610ed-d1ce-43ab-b217-92ba9367890d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:26.920538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9421f2a8299540af9c5dfca2965c578f', 'request_id': 'req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.950630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] The token cache is being invalidate because project 9421f2a8299540af9c5dfca2965c578f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:26.951989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.952889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29a4ca0d-af38-4407-98dd-ed1ee348e1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:26.960066 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1126/2228] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:26 2022] DELETE /identity/v3/projects/9421f2a8299540af9c5dfca2965c578f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:26.961778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0188b9-282a-48cd-ab8c-519c6aa425be None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:26.962074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0188b9-282a-48cd-ab8c-519c6aa425be None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:26.962385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0188b9-282a-48cd-ab8c-519c6aa425be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:26.993225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fb0188b9-282a-48cd-ab8c-519c6aa425be None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:26.995120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fb0188b9-282a-48cd-ab8c-519c6aa425be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:27.061754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:27.062706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.063389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.063845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.065911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:27.138006 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1127/2229] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:27.147145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908cf6-9cb0-43c3-a1a3-cf796a42f6f8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.147616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908cf6-9cb0-43c3-a1a3-cf796a42f6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.148128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908cf6-9cb0-43c3-a1a3-cf796a42f6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.181998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7c908cf6-9cb0-43c3-a1a3-cf796a42f6f8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:27.184193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7c908cf6-9cb0-43c3-a1a3-cf796a42f6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:27.190486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:27.196179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:27.197661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:27.217158 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1103/2230] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:27.220582 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1128/2231] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:27.225516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:27.239154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136069c-bc57-4f4e-a99f-c8d2b3812bbb service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:27.251555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:27.252184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.252465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.252894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.253448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:27.260209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:27.260209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:27.270088 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1104/2232] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:27.275135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c908cf6-9cb0-43c3-a1a3-cf796a42f6f8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:27.290964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c908cf6-9cb0-43c3-a1a3-cf796a42f6f8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:27.306261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a1eb-8e1b-4cbb-9cb8-aa058d9ae48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:27.307006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a1eb-8e1b-4cbb-9cb8-aa058d9ae48b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.307380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a1eb-8e1b-4cbb-9cb8-aa058d9ae48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.307748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9a1eb-8e1b-4cbb-9cb8-aa058d9ae48b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.308933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9a1eb-8e1b-4cbb-9cb8-aa058d9ae48b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:27.311995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9a1eb-8e1b-4cbb-9cb8-aa058d9ae48b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:27.321183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:27.324150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:27.345443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:27.364859 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1129/2233] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:27 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:27.369420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a1eb-8e1b-4cbb-9cb8-aa058d9ae48b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:27.448493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4c681-e448-4ccf-9ddf-f5b6014ed10d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:27.525793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:27.526769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.527116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.527456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.529539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:27.576056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9a1eb-8e1b-4cbb-9cb8-aa058d9ae48b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:27.583121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33201b8f-56e6-418c-a3a4-d86a789c2736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:27.584199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33201b8f-56e6-418c-a3a4-d86a789c2736 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.584729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33201b8f-56e6-418c-a3a4-d86a789c2736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.585349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33201b8f-56e6-418c-a3a4-d86a789c2736 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.586683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33201b8f-56e6-418c-a3a4-d86a789c2736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:27.591733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33201b8f-56e6-418c-a3a4-d86a789c2736 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:27.634035 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1130/2234] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:27 2022] POST /identity/v3/users => generated 287 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:27.638406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33201b8f-56e6-418c-a3a4-d86a789c2736 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:27.665586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33201b8f-56e6-418c-a3a4-d86a789c2736 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:27.666406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:27.667950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:27.673552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75972470-357d-4117-824d-794826d7f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:27.674594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972470-357d-4117-824d-794826d7f75a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.674594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972470-357d-4117-824d-794826d7f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.674783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972470-357d-4117-824d-794826d7f75a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.675436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75972470-357d-4117-824d-794826d7f75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:27.679498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75972470-357d-4117-824d-794826d7f75a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:27.696802 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1105/2235] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:27.748237 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1131/2236] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:27.748554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:27.766729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c8a5-4e6d-4635-bda0-7c0778d2a7e9 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:27.778111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcce1a-0f53-481c-9424-feae174d3ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:27.779338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcce1a-0f53-481c-9424-feae174d3ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.779709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcce1a-0f53-481c-9424-feae174d3ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.780053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcce1a-0f53-481c-9424-feae174d3ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.781017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcce1a-0f53-481c-9424-feae174d3ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:27.782947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcce1a-0f53-481c-9424-feae174d3ca5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:27.807427 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1106/2237] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:27.811666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75972470-357d-4117-824d-794826d7f75a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:27.830717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75972470-357d-4117-824d-794826d7f75a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:27.842001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:27.845194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.846322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.847363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] PATH_INFO: `/v3/projects/0124ee0c926240e8b69b3b70ef93080b/users/9f38a5ac076c4e9d841095bdca705c14/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.888440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0124ee0c926240e8b69b3b70ef93080b, user_id=9f38a5ac076c4e9d841095bdca705c14, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:27.893412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:27.928515 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1132/2238] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:27 2022] PUT /identity/v3/projects/0124ee0c926240e8b69b3b70ef93080b/users/9f38a5ac076c4e9d841095bdca705c14/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:27.940233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8835b-65fd-49df-aef8-e15b8eab381f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:27.940716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8835b-65fd-49df-aef8-e15b8eab381f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:27.941064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8835b-65fd-49df-aef8-e15b8eab381f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:27.994436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-58c8835b-65fd-49df-aef8-e15b8eab381f None None] MFA Rules not processed for user `9f38a5ac076c4e9d841095bdca705c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:27.995725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-58c8835b-65fd-49df-aef8-e15b8eab381f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:28.049838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.055182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:28.069142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:28.182778 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1107/2239] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:28.234987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8835b-65fd-49df-aef8-e15b8eab381f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.236413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-58c8835b-65fd-49df-aef8-e15b8eab381f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:28.238092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51d27e-58fe-4de9-9bdc-79df326bb702 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:28.249495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:28.251129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.251461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.251739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] PATH_INFO: `/v3/projects/b139144c40074965b405ddd5acdfbbd3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.254701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b139144c40074965b405ddd5acdfbbd3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:28.257868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:28.345920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8835b-65fd-49df-aef8-e15b8eab381f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:28.370367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b139144c40074965b405ddd5acdfbbd3', 'request_id': 'req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:28.377877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:28.387536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b139144c40074965b405ddd5acdfbbd3', 'request_id': 'req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:28.405310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b139144c40074965b405ddd5acdfbbd3', 'request_id': 'req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:28.442442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] The token cache is being invalidate because project b139144c40074965b405ddd5acdfbbd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:28.442766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:28.443687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8562a1e8-b1b5-471f-ab8e-83ea2431c8d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:28.446670 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1133/2240] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:28 2022] DELETE /identity/v3/projects/b139144c40074965b405ddd5acdfbbd3 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:28.456555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484aa66-1691-4db2-8474-2490144a9e2f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.456555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484aa66-1691-4db2-8474-2490144a9e2f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.456965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484aa66-1691-4db2-8474-2490144a9e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.483177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7484aa66-1691-4db2-8474-2490144a9e2f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:28.484733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7484aa66-1691-4db2-8474-2490144a9e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:28.485716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:28.486306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.486599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.487016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.490025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:28.574134 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1134/2241] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:28.584044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83725f-1efc-4dbc-954b-1c181d827205 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.584180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:28.584416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83725f-1efc-4dbc-954b-1c181d827205 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.584764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83725f-1efc-4dbc-954b-1c181d827205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.585959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:28.609952 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1108/2242] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:28.617973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5a83725f-1efc-4dbc-954b-1c181d827205 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:28.619413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5a83725f-1efc-4dbc-954b-1c181d827205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:28.635934 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1135/2243] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:28.637345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.650133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2228-6fb0-47f3-8f78-c77339562c1a service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:28.668204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:28.668204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.668204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.668204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.668573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:28.668573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:28.668573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:28.677611 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1109/2244] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:37:28.682464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83725f-1efc-4dbc-954b-1c181d827205 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.701019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83725f-1efc-4dbc-954b-1c181d827205 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:28.706970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:28.707717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.708098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.709095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.723045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:28.725528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:28.769687 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1136/2245] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:28 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:28.773490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.793946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89883d2a-7a99-4749-94d4-0456fe719a6f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:28.821829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.823599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:28.835414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:28.873901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a1536-5784-474e-9e5e-b2da8927f6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:28.874598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1536-5784-474e-9e5e-b2da8927f6db admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.875055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1536-5784-474e-9e5e-b2da8927f6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.876047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1536-5784-474e-9e5e-b2da8927f6db admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.876703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a1536-5784-474e-9e5e-b2da8927f6db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:28.879588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a1536-5784-474e-9e5e-b2da8927f6db admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:28.897054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e2c9a-8bf8-47a3-81b3-69921d5a6fa0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:28.902024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:28.902657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.902934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.903261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.904468 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1110/2246] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:28 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:28.906626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:28.908423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a1536-5784-474e-9e5e-b2da8927f6db admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.919505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a1536-5784-474e-9e5e-b2da8927f6db admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:28.924319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced4091-a305-4bc9-a360-59a5433b88db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:28.924930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced4091-a305-4bc9-a360-59a5433b88db admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.925211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced4091-a305-4bc9-a360-59a5433b88db admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.926157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced4091-a305-4bc9-a360-59a5433b88db admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.927032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced4091-a305-4bc9-a360-59a5433b88db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:28.928745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced4091-a305-4bc9-a360-59a5433b88db admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:28.942538 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1111/2247] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:28.947776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced4091-a305-4bc9-a360-59a5433b88db admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.961039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced4091-a305-4bc9-a360-59a5433b88db admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:28.966686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5af5b-e97f-471d-86d7-95cbc9ec0777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:28.967747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5af5b-e97f-471d-86d7-95cbc9ec0777 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:28.967866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5af5b-e97f-471d-86d7-95cbc9ec0777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:28.968226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5af5b-e97f-471d-86d7-95cbc9ec0777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:28.971747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5af5b-e97f-471d-86d7-95cbc9ec0777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:28.971747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5af5b-e97f-471d-86d7-95cbc9ec0777 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:28.983262 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1112/2248] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:28.987536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5af5b-e97f-471d-86d7-95cbc9ec0777 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:28.999818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5af5b-e97f-471d-86d7-95cbc9ec0777 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.011058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dfb59-97fa-4b56-81c6-71104ceb11b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.011733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dfb59-97fa-4b56-81c6-71104ceb11b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.012024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dfb59-97fa-4b56-81c6-71104ceb11b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.012299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dfb59-97fa-4b56-81c6-71104ceb11b0 admin admin] PATH_INFO: `/v3/projects/6f2c247f903a4f688450629a5fc7d6ca/users/dca4915b52e043aea7debb5ddde9d4a6/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.032680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.034296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.039170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dfb59-97fa-4b56-81c6-71104ceb11b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2c247f903a4f688450629a5fc7d6ca, user_id=dca4915b52e043aea7debb5ddde9d4a6, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.041904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dfb59-97fa-4b56-81c6-71104ceb11b0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.046122 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1137/2249] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:29.046906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.048525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:29.057193 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1113/2250] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:28 2022] PUT /identity/v3/projects/6f2c247f903a4f688450629a5fc7d6ca/users/dca4915b52e043aea7debb5ddde9d4a6/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:29.059217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:29.067644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0028dcd-dfe0-4851-91db-83fdeb8a2e84 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.068670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc4c3e-c557-4ac9-9835-eddaae6927be None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.068969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc4c3e-c557-4ac9-9835-eddaae6927be None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.069355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc4c3e-c557-4ac9-9835-eddaae6927be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.103066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ffdc4c3e-c557-4ac9-9835-eddaae6927be None None] MFA Rules not processed for user `dca4915b52e043aea7debb5ddde9d4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:29.104416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ffdc4c3e-c557-4ac9-9835-eddaae6927be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:29.138617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.139563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.139964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.140388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] PATH_INFO: `/v3/users/a1d3816451a8467f9afe467b8866b53d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.142226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d3816451a8467f9afe467b8866b53d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.144218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.198298 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1114/2251] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:29.198693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d3816451a8467f9afe467b8866b53d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.202623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc4c3e-c557-4ac9-9835-eddaae6927be None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.205891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d3816451a8467f9afe467b8866b53d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.206561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d3816451a8467f9afe467b8866b53d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.207291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d3816451a8467f9afe467b8866b53d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.213089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc4c3e-c557-4ac9-9835-eddaae6927be None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.215367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bd845b4a-8e5c-4441-9e90-1f40142e4c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d3816451a8467f9afe467b8866b53d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.218118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.218724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.219001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.219270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.219899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.223150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.234656 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1138/2252] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:29 2022] DELETE /identity/v3/users/a1d3816451a8467f9afe467b8866b53d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:29.239690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cee95-e567-434f-8beb-8205bf2030a2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.240050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cee95-e567-434f-8beb-8205bf2030a2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.240405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cee95-e567-434f-8beb-8205bf2030a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.260228 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1115/2253] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:29 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:29.264046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.265150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-594cee95-e567-434f-8beb-8205bf2030a2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:29.266231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-594cee95-e567-434f-8beb-8205bf2030a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:29.266350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:29.324858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:29.334957 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1139/2254] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:29.350043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cee95-e567-434f-8beb-8205bf2030a2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.351432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-594cee95-e567-434f-8beb-8205bf2030a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:29.356183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab0aa-6f66-49bd-82fe-f9f7947dc26b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.361731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b57223-cb3e-4c5d-b2f2-643ca517e664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.362409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57223-cb3e-4c5d-b2f2-643ca517e664 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.362757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57223-cb3e-4c5d-b2f2-643ca517e664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.363616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57223-cb3e-4c5d-b2f2-643ca517e664 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.364915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b57223-cb3e-4c5d-b2f2-643ca517e664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.366344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b57223-cb3e-4c5d-b2f2-643ca517e664 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.382668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:29.398128 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1116/2255] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:29 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:29.401247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b57223-cb3e-4c5d-b2f2-643ca517e664 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.428518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cee95-e567-434f-8beb-8205bf2030a2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.432508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b57223-cb3e-4c5d-b2f2-643ca517e664 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.432880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00da99d-e490-4aba-ba7e-a22c5281ccc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.433519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00da99d-e490-4aba-ba7e-a22c5281ccc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.434259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00da99d-e490-4aba-ba7e-a22c5281ccc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.434544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00da99d-e490-4aba-ba7e-a22c5281ccc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.439361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a8c67-836b-45ad-935a-ed1783778d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.439988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8c67-836b-45ad-935a-ed1783778d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.440283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8c67-836b-45ad-935a-ed1783778d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.440546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a8c67-836b-45ad-935a-ed1783778d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.441591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a8c67-836b-45ad-935a-ed1783778d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.444032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a8c67-836b-45ad-935a-ed1783778d9c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.446817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00da99d-e490-4aba-ba7e-a22c5281ccc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.448958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00da99d-e490-4aba-ba7e-a22c5281ccc3 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.455789 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1117/2256] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:29.460364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a8c67-836b-45ad-935a-ed1783778d9c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.463168 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1140/2257] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:29.475505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a8c67-836b-45ad-935a-ed1783778d9c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.485950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa6c9-b41c-4aa0-8025-ccb39419b8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.486702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa6c9-b41c-4aa0-8025-ccb39419b8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.487054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa6c9-b41c-4aa0-8025-ccb39419b8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.487398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa6c9-b41c-4aa0-8025-ccb39419b8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.488765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa6c9-b41c-4aa0-8025-ccb39419b8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.491544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa6c9-b41c-4aa0-8025-ccb39419b8a9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.506409 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1118/2258] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:29.507787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00da99d-e490-4aba-ba7e-a22c5281ccc3 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.518114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00da99d-e490-4aba-ba7e-a22c5281ccc3 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.523493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b0393-4b22-4f15-a750-efd825956820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.524154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0393-4b22-4f15-a750-efd825956820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.524428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0393-4b22-4f15-a750-efd825956820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.524705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0393-4b22-4f15-a750-efd825956820 admin admin] PATH_INFO: `/v3/projects/23e6a3dd20c7484b8946bb73bbdd7b2c/users/0f0816c367aa42d19ce577c1f8ce5b49/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.540584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b0393-4b22-4f15-a750-efd825956820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e6a3dd20c7484b8946bb73bbdd7b2c, user_id=0f0816c367aa42d19ce577c1f8ce5b49, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.543320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b0393-4b22-4f15-a750-efd825956820 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.557190 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1141/2259] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:29 2022] PUT /identity/v3/projects/23e6a3dd20c7484b8946bb73bbdd7b2c/users/0f0816c367aa42d19ce577c1f8ce5b49/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:29.569611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67d61-02b3-4764-b5ac-e5a0c1f4eb86 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.569907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67d61-02b3-4764-b5ac-e5a0c1f4eb86 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.570237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f67d61-02b3-4764-b5ac-e5a0c1f4eb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.605843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-73f67d61-02b3-4764-b5ac-e5a0c1f4eb86 None None] MFA Rules not processed for user `0f0816c367aa42d19ce577c1f8ce5b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:29.607092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-73f67d61-02b3-4764-b5ac-e5a0c1f4eb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:29.681296 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1119/2260] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:29.686059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b0393-4b22-4f15-a750-efd825956820 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.749754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f67d61-02b3-4764-b5ac-e5a0c1f4eb86 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.751214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-73f67d61-02b3-4764-b5ac-e5a0c1f4eb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:29.765127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b0393-4b22-4f15-a750-efd825956820 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.771493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336f88-d943-4a28-a98d-a880ec195ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.772225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336f88-d943-4a28-a98d-a880ec195ced admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.772460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336f88-d943-4a28-a98d-a880ec195ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.772806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336f88-d943-4a28-a98d-a880ec195ced admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.773510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78336f88-d943-4a28-a98d-a880ec195ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.775618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78336f88-d943-4a28-a98d-a880ec195ced admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.780497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f67d61-02b3-4764-b5ac-e5a0c1f4eb86 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.785552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.786375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.786657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.786932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] PATH_INFO: `/v3/projects/3a634055eea14346bd2ec2e14151633b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.788589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a634055eea14346bd2ec2e14151633b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.801072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:29.809181 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1142/2261] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:29 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:29.811405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336f88-d943-4a28-a98d-a880ec195ced admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:29.876814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a634055eea14346bd2ec2e14151633b', 'request_id': 'req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.884452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a634055eea14346bd2ec2e14151633b', 'request_id': 'req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.891881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a634055eea14346bd2ec2e14151633b', 'request_id': 'req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.901475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336f88-d943-4a28-a98d-a880ec195ced admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:29.903621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] The token cache is being invalidate because project 3a634055eea14346bd2ec2e14151633b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:29.903916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.904763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c1e9c88f-1e7e-4f6b-bfaf-523b2501137d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:29.906560 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1120/2262] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:29 2022] DELETE /identity/v3/projects/3a634055eea14346bd2ec2e14151633b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:29.912211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe469af-82c7-423a-af45-0582e9b8658f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.913614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe469af-82c7-423a-af45-0582e9b8658f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.913925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe469af-82c7-423a-af45-0582e9b8658f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.937255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fbe469af-82c7-423a-af45-0582e9b8658f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:29.938623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fbe469af-82c7-423a-af45-0582e9b8658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:29.944818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:29.959321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:29.984416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:29.985375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:29.985681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:29.985956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:29.986623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:29.989651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.017731 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1143/2263] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:29 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:30.020632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.022532 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1121/2264] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:30.023018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:30.030046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e40d-d735-4746-a1d5-df9a3279099d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.030251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e40d-d735-4746-a1d5-df9a3279099d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.030596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e40d-d735-4746-a1d5-df9a3279099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.037421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ffee-6213-44a4-9677-0c5aea980d32 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.044065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f54c4d-633e-490c-b795-1343460f4c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.044750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f54c4d-633e-490c-b795-1343460f4c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.057654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f54c4d-633e-490c-b795-1343460f4c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.057654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f54c4d-633e-490c-b795-1343460f4c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.057654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f54c4d-633e-490c-b795-1343460f4c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.057654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f54c4d-633e-490c-b795-1343460f4c90 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.062427 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1144/2265] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:30.065570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f54c4d-633e-490c-b795-1343460f4c90 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.077250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f54c4d-633e-490c-b795-1343460f4c90 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.090200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d6f6e40d-d735-4746-a1d5-df9a3279099d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:30.090200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d6f6e40d-d735-4746-a1d5-df9a3279099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:30.095731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca1dab-0e10-4eee-aaea-7512ef0051bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.096327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca1dab-0e10-4eee-aaea-7512ef0051bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.098072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca1dab-0e10-4eee-aaea-7512ef0051bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.098507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca1dab-0e10-4eee-aaea-7512ef0051bf admin admin] PATH_INFO: `/v3/projects/fc82cf17bf52476fa03ff9cb83713361/users/51998ac1d5d24c069ef23f61d778ac53/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.102598 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1122/2266] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:30.108167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf0f53-abee-49f0-812f-d313d67f4239 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.109068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf0f53-abee-49f0-812f-d313d67f4239 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.109271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf0f53-abee-49f0-812f-d313d67f4239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.123428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca1dab-0e10-4eee-aaea-7512ef0051bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc82cf17bf52476fa03ff9cb83713361, user_id=51998ac1d5d24c069ef23f61d778ac53, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.126341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca1dab-0e10-4eee-aaea-7512ef0051bf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.143467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-47bf0f53-abee-49f0-812f-d313d67f4239 None None] MFA Rules not processed for user `9f38a5ac076c4e9d841095bdca705c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:30.145891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-47bf0f53-abee-49f0-812f-d313d67f4239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:30.150542 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1145/2267] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:30 2022] PUT /identity/v3/projects/fc82cf17bf52476fa03ff9cb83713361/users/51998ac1d5d24c069ef23f61d778ac53/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:30.154227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca1dab-0e10-4eee-aaea-7512ef0051bf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.220600 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1123/2268] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:30.221250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf0f53-abee-49f0-812f-d313d67f4239 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.240029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca1dab-0e10-4eee-aaea-7512ef0051bf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.245266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.246162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.246593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.246961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.247817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:30.248353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.250444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.264313 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1146/2269] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:30.268500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.280861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713af48-d99b-4477-a5db-e3d32d5ca8c2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.287726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265497b7-bbb3-4d30-9a22-e324659c5d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.288605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265497b7-bbb3-4d30-9a22-e324659c5d77 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.289750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265497b7-bbb3-4d30-9a22-e324659c5d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.289750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265497b7-bbb3-4d30-9a22-e324659c5d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.290353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265497b7-bbb3-4d30-9a22-e324659c5d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.292522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265497b7-bbb3-4d30-9a22-e324659c5d77 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.308119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf0f53-abee-49f0-812f-d313d67f4239 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.314408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd3862-5525-4860-abd4-0f08c3ff461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.315267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd3862-5525-4860-abd4-0f08c3ff461b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.315769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd3862-5525-4860-abd4-0f08c3ff461b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.316129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd3862-5525-4860-abd4-0f08c3ff461b admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.317823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd3862-5525-4860-abd4-0f08c3ff461b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.319453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd3862-5525-4860-abd4-0f08c3ff461b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.330990 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1124/2270] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:30.333045 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1147/2271] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:30 2022] POST /identity/v3/projects => generated 361 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:30.337380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265497b7-bbb3-4d30-9a22-e324659c5d77 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.337513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd3862-5525-4860-abd4-0f08c3ff461b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.352117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd3862-5525-4860-abd4-0f08c3ff461b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.443360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265497b7-bbb3-4d30-9a22-e324659c5d77 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.450843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:30.451131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e73e5-390d-4abf-8a0b-897f0d2807d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.452591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e73e5-390d-4abf-8a0b-897f0d2807d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.453120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e73e5-390d-4abf-8a0b-897f0d2807d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.453400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e73e5-390d-4abf-8a0b-897f0d2807d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.454652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e73e5-390d-4abf-8a0b-897f0d2807d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.456898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e73e5-390d-4abf-8a0b-897f0d2807d5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.461555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a304f0-06fa-4090-9e35-5d8d157ff999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.462382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a304f0-06fa-4090-9e35-5d8d157ff999 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.462774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a304f0-06fa-4090-9e35-5d8d157ff999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.463145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a304f0-06fa-4090-9e35-5d8d157ff999 admin admin] PATH_INFO: `/v3/projects/fc82cf17bf52476fa03ff9cb83713361/users/51998ac1d5d24c069ef23f61d778ac53/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.465614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a304f0-06fa-4090-9e35-5d8d157ff999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc82cf17bf52476fa03ff9cb83713361, user_id=51998ac1d5d24c069ef23f61d778ac53, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.468723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:30.469687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a304f0-06fa-4090-9e35-5d8d157ff999 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.497937 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1125/2272] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:30 2022] PUT /identity/v3/projects/fc82cf17bf52476fa03ff9cb83713361/users/51998ac1d5d24c069ef23f61d778ac53/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:30.499947 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1148/2273] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:30 2022] POST /identity/v3/users => generated 281 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:30.511151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a304f0-06fa-4090-9e35-5d8d157ff999 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.514977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352657f-ae30-4391-881d-f41822b0cb18 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.515264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352657f-ae30-4391-881d-f41822b0cb18 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.515539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352657f-ae30-4391-881d-f41822b0cb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.598364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7352657f-ae30-4391-881d-f41822b0cb18 None None] MFA Rules not processed for user `51998ac1d5d24c069ef23f61d778ac53`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:30.599973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7352657f-ae30-4391-881d-f41822b0cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:30.605186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a304f0-06fa-4090-9e35-5d8d157ff999 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.627817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb90a55-726a-4751-9ce5-809f6c2b78f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.628796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb90a55-726a-4751-9ce5-809f6c2b78f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.630127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb90a55-726a-4751-9ce5-809f6c2b78f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.630501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb90a55-726a-4751-9ce5-809f6c2b78f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.631261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb90a55-726a-4751-9ce5-809f6c2b78f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.632961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb90a55-726a-4751-9ce5-809f6c2b78f4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.645853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1126/2274] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:30.651663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb90a55-726a-4751-9ce5-809f6c2b78f4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.664140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb90a55-726a-4751-9ce5-809f6c2b78f4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.671453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b066-9320-4c27-a95c-202eac3be3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.672205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b066-9320-4c27-a95c-202eac3be3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.674062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b066-9320-4c27-a95c-202eac3be3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.674062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b066-9320-4c27-a95c-202eac3be3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.674062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3b066-9320-4c27-a95c-202eac3be3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.676822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3b066-9320-4c27-a95c-202eac3be3ba admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.690088 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1127/2275] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:30.695028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b066-9320-4c27-a95c-202eac3be3ba admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.711144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b066-9320-4c27-a95c-202eac3be3ba admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.717968 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1149/2276] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:30.720150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6ec03-478f-4add-9cfd-2ff3e9505290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.721190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6ec03-478f-4add-9cfd-2ff3e9505290 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.721570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6ec03-478f-4add-9cfd-2ff3e9505290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.725236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6ec03-478f-4add-9cfd-2ff3e9505290 admin admin] PATH_INFO: `/v3/projects/dd24382898d5474a86e7bb4524d1b007/users/48e7ffacb5f247fdb9441468b3943acf/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.730228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352657f-ae30-4391-881d-f41822b0cb18 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:30.732692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7352657f-ae30-4391-881d-f41822b0cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:30.748808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6ec03-478f-4add-9cfd-2ff3e9505290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd24382898d5474a86e7bb4524d1b007, user_id=48e7ffacb5f247fdb9441468b3943acf, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:30.751887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6ec03-478f-4add-9cfd-2ff3e9505290 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:30.774103 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1128/2277] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:30 2022] PUT /identity/v3/projects/dd24382898d5474a86e7bb4524d1b007/users/48e7ffacb5f247fdb9441468b3943acf/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:30.789257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3744420-7c75-4889-be99-ea0726f28f7e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.789711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3744420-7c75-4889-be99-ea0726f28f7e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.790265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3744420-7c75-4889-be99-ea0726f28f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.825840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352657f-ae30-4391-881d-f41822b0cb18 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:30.826721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d3744420-7c75-4889-be99-ea0726f28f7e None None] MFA Rules not processed for user `48e7ffacb5f247fdb9441468b3943acf`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:30.828120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d3744420-7c75-4889-be99-ea0726f28f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:30.912242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:30.916042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:30.916572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:30.916928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:30.922176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:30.995873 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1129/2278] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:31.022167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:31.052567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3744420-7c75-4889-be99-ea0726f28f7e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:31.054164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d3744420-7c75-4889-be99-ea0726f28f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:31.069108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:31.071544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:31.073227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3744420-7c75-4889-be99-ea0726f28f7e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:31.078344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1257-0314-438e-b52f-d3c4c7bef8b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:31.079215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1257-0314-438e-b52f-d3c4c7bef8b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:31.079763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1257-0314-438e-b52f-d3c4c7bef8b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:31.080238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1257-0314-438e-b52f-d3c4c7bef8b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:31.092783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1257-0314-438e-b52f-d3c4c7bef8b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:31.094651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1257-0314-438e-b52f-d3c4c7bef8b7 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:31.100272 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1150/2279] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:31.121971 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1130/2280] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:31.131232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:31.210775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09442d-45fd-4181-911b-ce7aedba3435 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:31.216969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:31.218131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:31.218397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:31.218859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] PATH_INFO: `/v3/users/48e7ffacb5f247fdb9441468b3943acf` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:31.220530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e7ffacb5f247fdb9441468b3943acf)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:31.223657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:31.398969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e7ffacb5f247fdb9441468b3943acf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:31.428136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e7ffacb5f247fdb9441468b3943acf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:31.429487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e7ffacb5f247fdb9441468b3943acf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:31.431599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e7ffacb5f247fdb9441468b3943acf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:31.445920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e7ffacb5f247fdb9441468b3943acf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:31.479038 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1151/2281] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:31 2022] DELETE /identity/v3/users/48e7ffacb5f247fdb9441468b3943acf => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:31.484005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148f18e-3077-4e8c-a699-35a63f57c251 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:31.484175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148f18e-3077-4e8c-a699-35a63f57c251 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:31.484981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148f18e-3077-4e8c-a699-35a63f57c251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:31.527430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3148f18e-3077-4e8c-a699-35a63f57c251 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:31.531134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3148f18e-3077-4e8c-a699-35a63f57c251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:31.628541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:31.814682 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1131/2282] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:31.841323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:31.843413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:31.995057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0addb6-bafa-4180-a075-64efae3987be admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:32.004037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d964fe-b920-4457-90b3-7afce2781136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:32.004037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d964fe-b920-4457-90b3-7afce2781136 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:32.004037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d964fe-b920-4457-90b3-7afce2781136 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:32.004037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d964fe-b920-4457-90b3-7afce2781136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:32.020146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d964fe-b920-4457-90b3-7afce2781136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:32.022346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d964fe-b920-4457-90b3-7afce2781136 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:32.039257 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1152/2283] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:32.494266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148f18e-3077-4e8c-a699-35a63f57c251 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:32.495734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3148f18e-3077-4e8c-a699-35a63f57c251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:32.509925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148f18e-3077-4e8c-a699-35a63f57c251 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:32.514696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:32.515418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:32.516769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:32.517052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] PATH_INFO: `/v3/projects/dd24382898d5474a86e7bb4524d1b007` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:32.518295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd24382898d5474a86e7bb4524d1b007)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:32.520563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:32.570156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd24382898d5474a86e7bb4524d1b007', 'request_id': 'req-60823bd7-65ef-4d6a-a879-b77c1e843cae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:32.577579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd24382898d5474a86e7bb4524d1b007', 'request_id': 'req-60823bd7-65ef-4d6a-a879-b77c1e843cae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:32.585107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd24382898d5474a86e7bb4524d1b007', 'request_id': 'req-60823bd7-65ef-4d6a-a879-b77c1e843cae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:32.596096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] The token cache is being invalidate because project dd24382898d5474a86e7bb4524d1b007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:32.596386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:32.597026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-60823bd7-65ef-4d6a-a879-b77c1e843cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:32.599545 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1132/2284] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:32 2022] DELETE /identity/v3/projects/dd24382898d5474a86e7bb4524d1b007 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:32.605328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e15b3-d177-41a7-8fca-b316cf901391 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:32.605668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e15b3-d177-41a7-8fca-b316cf901391 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:32.606267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e15b3-d177-41a7-8fca-b316cf901391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:32.628924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-957e15b3-d177-41a7-8fca-b316cf901391 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:32.632214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-957e15b3-d177-41a7-8fca-b316cf901391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:32.634979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:32.709166 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1153/2285] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:32.717945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae02c-979b-4f35-a6ef-08e693173726 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:32.718221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae02c-979b-4f35-a6ef-08e693173726 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:32.718589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae02c-979b-4f35-a6ef-08e693173726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:32.741391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f3bae02c-979b-4f35-a6ef-08e693173726 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:32.742859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f3bae02c-979b-4f35-a6ef-08e693173726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:32.760066 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1133/2286] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:32.760253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e15b3-d177-41a7-8fca-b316cf901391 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:32.770722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e15b3-d177-41a7-8fca-b316cf901391 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:32.777813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:32.778429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:32.778719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:32.778968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:32.779615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:32.779920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:32.781571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:32.798163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1154/2287] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:32.803121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bae02c-979b-4f35-a6ef-08e693173726 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:32.814710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bae02c-979b-4f35-a6ef-08e693173726 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:32.822372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:32.823001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:32.823286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:32.823558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:32.824252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:32.826439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:32.870445 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1134/2288] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:32 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:32.873573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:32.889989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a95e3-70d9-45ee-b3ab-f03513ec0440 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:32.977664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:32.980087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:33.011910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98884-f5cd-4496-9215-c60e14d83003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:33.012807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98884-f5cd-4496-9215-c60e14d83003 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.013258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98884-f5cd-4496-9215-c60e14d83003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.013727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98884-f5cd-4496-9215-c60e14d83003 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.014530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e98884-f5cd-4496-9215-c60e14d83003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:33.020225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e98884-f5cd-4496-9215-c60e14d83003 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:33.078695 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1155/2289] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:32 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:33.079706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98884-f5cd-4496-9215-c60e14d83003 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:33.102102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98884-f5cd-4496-9215-c60e14d83003 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:33.113547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085edcf-685b-42fd-8ed1-abbfad9fb1a0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:33.116433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bfd96-7cf0-416d-8a9f-2b73822ea889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:33.117203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bfd96-7cf0-416d-8a9f-2b73822ea889 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.117489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bfd96-7cf0-416d-8a9f-2b73822ea889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.117804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bfd96-7cf0-416d-8a9f-2b73822ea889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.118480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bfd96-7cf0-416d-8a9f-2b73822ea889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:33.120300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bfd96-7cf0-416d-8a9f-2b73822ea889 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:33.126886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:33.127519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.127839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.128151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.135020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:33.153246 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1156/2290] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:33.154752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bfd96-7cf0-416d-8a9f-2b73822ea889 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:33.181213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bfd96-7cf0-416d-8a9f-2b73822ea889 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:33.186670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e2b2-7d92-4d4a-adec-baadd0f63d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:33.187295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e2b2-7d92-4d4a-adec-baadd0f63d52 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.187688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e2b2-7d92-4d4a-adec-baadd0f63d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.187969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23e2b2-7d92-4d4a-adec-baadd0f63d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.188619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23e2b2-7d92-4d4a-adec-baadd0f63d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:33.190373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23e2b2-7d92-4d4a-adec-baadd0f63d52 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:33.205241 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1157/2291] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:33.207191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e2b2-7d92-4d4a-adec-baadd0f63d52 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:33.218173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23e2b2-7d92-4d4a-adec-baadd0f63d52 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:33.226120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571f431-d753-4f20-b3bf-de7b4853be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:33.227330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571f431-d753-4f20-b3bf-de7b4853be21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.229114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571f431-d753-4f20-b3bf-de7b4853be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.229402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571f431-d753-4f20-b3bf-de7b4853be21 admin admin] PATH_INFO: `/v3/projects/2a168585ae4149f58a32da9be0b1878e/users/75e2aec5891f4b519623680718aa1d28/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.239009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:33.276774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571f431-d753-4f20-b3bf-de7b4853be21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a168585ae4149f58a32da9be0b1878e, user_id=75e2aec5891f4b519623680718aa1d28, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:33.284189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571f431-d753-4f20-b3bf-de7b4853be21 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:33.310818 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1158/2292] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:33 2022] PUT /identity/v3/projects/2a168585ae4149f58a32da9be0b1878e/users/75e2aec5891f4b519623680718aa1d28/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:33.335908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b217a-1e79-4e8f-a850-bd0799ce7f56 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.335908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b217a-1e79-4e8f-a850-bd0799ce7f56 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.335908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b217a-1e79-4e8f-a850-bd0799ce7f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.345484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:33.350743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:33.422193 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1135/2293] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:33.454385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-262b217a-1e79-4e8f-a850-bd0799ce7f56 None None] MFA Rules not processed for user `75e2aec5891f4b519623680718aa1d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:33.456811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-262b217a-1e79-4e8f-a850-bd0799ce7f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:33.555051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:33.611875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:33.614116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:33.638091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0ebe-64e8-4146-8bf8-b4df39bbbf92 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:33.644015 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1159/2294] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:33.644972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b217a-1e79-4e8f-a850-bd0799ce7f56 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:33.723361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:33.724338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.724524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.724789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] PATH_INFO: `/v3/users/ffd4fd8a8ad045e099c7be28a814c7d4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.726059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd4fd8a8ad045e099c7be28a814c7d4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:33.728412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:33.733313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b217a-1e79-4e8f-a850-bd0799ce7f56 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:33.738850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59662833-b0ea-4307-b871-22ff6fae546d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:33.739952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662833-b0ea-4307-b871-22ff6fae546d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.740236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662833-b0ea-4307-b871-22ff6fae546d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.740757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662833-b0ea-4307-b871-22ff6fae546d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.741537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59662833-b0ea-4307-b871-22ff6fae546d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:33.744817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59662833-b0ea-4307-b871-22ff6fae546d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:33.783280 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1160/2295] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:33 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:33.784618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59662833-b0ea-4307-b871-22ff6fae546d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:33.791830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd4fd8a8ad045e099c7be28a814c7d4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:33.799372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd4fd8a8ad045e099c7be28a814c7d4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:33.800192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd4fd8a8ad045e099c7be28a814c7d4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:33.800844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd4fd8a8ad045e099c7be28a814c7d4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:33.810550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-26cb2fb8-6846-49ed-bc92-6e4a1b3a8c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd4fd8a8ad045e099c7be28a814c7d4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:33.833965 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1136/2296] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:33 2022] DELETE /identity/v3/users/ffd4fd8a8ad045e099c7be28a814c7d4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:33.838526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33b79e-e33d-4f14-8ec1-23ac7d3ac27a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.838803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33b79e-e33d-4f14-8ec1-23ac7d3ac27a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.839091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33b79e-e33d-4f14-8ec1-23ac7d3ac27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.860514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ad33b79e-e33d-4f14-8ec1-23ac7d3ac27a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:33.860793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59662833-b0ea-4307-b871-22ff6fae546d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:33.862366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ad33b79e-e33d-4f14-8ec1-23ac7d3ac27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:33.938008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:33.947865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:33.948516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:33.948810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:33.949281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:33.950231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:33.952224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:33.955913 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1137/2297] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:33.967398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:33.974855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33b79e-e33d-4f14-8ec1-23ac7d3ac27a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:33.976484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ad33b79e-e33d-4f14-8ec1-23ac7d3ac27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:33.987481 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1161/2298] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:33 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:33.990553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:33.992743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:34.012204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820612a2-15fe-4081-8c67-848c9f4e89fd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.021500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0858e-6e48-438e-9496-abdb0fb69d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.022186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0858e-6e48-438e-9496-abdb0fb69d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.022470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0858e-6e48-438e-9496-abdb0fb69d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.022742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0858e-6e48-438e-9496-abdb0fb69d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.023407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0858e-6e48-438e-9496-abdb0fb69d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.027484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0858e-6e48-438e-9496-abdb0fb69d07 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.046190 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1162/2299] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:34.053893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0858e-6e48-438e-9496-abdb0fb69d07 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.072878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0858e-6e48-438e-9496-abdb0fb69d07 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.079967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1730d8-ebc7-4db1-8fed-5d2211710a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.080940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1730d8-ebc7-4db1-8fed-5d2211710a06 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.081345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1730d8-ebc7-4db1-8fed-5d2211710a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.082336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1730d8-ebc7-4db1-8fed-5d2211710a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.083291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1730d8-ebc7-4db1-8fed-5d2211710a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.084863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33b79e-e33d-4f14-8ec1-23ac7d3ac27a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.085720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1730d8-ebc7-4db1-8fed-5d2211710a06 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.090997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796ce3a-74e8-4de9-b4ce-53654be02bcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.092912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796ce3a-74e8-4de9-b4ce-53654be02bcb service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.093440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796ce3a-74e8-4de9-b4ce-53654be02bcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.093896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796ce3a-74e8-4de9-b4ce-53654be02bcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.101007 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1163/2300] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:34.106526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1730d8-ebc7-4db1-8fed-5d2211710a06 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.110056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796ce3a-74e8-4de9-b4ce-53654be02bcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.111516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796ce3a-74e8-4de9-b4ce-53654be02bcb service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.122729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1730d8-ebc7-4db1-8fed-5d2211710a06 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.124130 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1138/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:34.129842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6721dd-8789-4bda-bed5-ab22fd92788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.130628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6721dd-8789-4bda-bed5-ab22fd92788c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.130908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6721dd-8789-4bda-bed5-ab22fd92788c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.131182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6721dd-8789-4bda-bed5-ab22fd92788c admin admin] PATH_INFO: `/v3/projects/e9a0fc8336054e42811fe6ba43ac20dd/users/d7802e5aa89c464a9313a338a76f38bf/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.151697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6721dd-8789-4bda-bed5-ab22fd92788c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a0fc8336054e42811fe6ba43ac20dd, user_id=d7802e5aa89c464a9313a338a76f38bf, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.155346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6721dd-8789-4bda-bed5-ab22fd92788c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.179507 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1164/2302] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:34 2022] PUT /identity/v3/projects/e9a0fc8336054e42811fe6ba43ac20dd/users/d7802e5aa89c464a9313a338a76f38bf/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:34.192438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a9c62-d7f6-430c-ad5b-7a0826af5266 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.192816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a9c62-d7f6-430c-ad5b-7a0826af5266 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.192989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a9c62-d7f6-430c-ad5b-7a0826af5266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.250056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cc4a9c62-d7f6-430c-ad5b-7a0826af5266 None None] MFA Rules not processed for user `d7802e5aa89c464a9313a338a76f38bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:34.252180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cc4a9c62-d7f6-430c-ad5b-7a0826af5266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:34.365279 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1139/2303] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:34.369758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6721dd-8789-4bda-bed5-ab22fd92788c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.413893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a9c62-d7f6-430c-ad5b-7a0826af5266 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.415787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cc4a9c62-d7f6-430c-ad5b-7a0826af5266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:34.444580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6721dd-8789-4bda-bed5-ab22fd92788c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.450080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3738a88-0db5-4217-907c-196fd9445672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.450748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738a88-0db5-4217-907c-196fd9445672 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.451031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738a88-0db5-4217-907c-196fd9445672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.451575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3738a88-0db5-4217-907c-196fd9445672 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.452415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3738a88-0db5-4217-907c-196fd9445672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.454920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3738a88-0db5-4217-907c-196fd9445672 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.483558 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1165/2304] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:34 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:34.484877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a9c62-d7f6-430c-ad5b-7a0826af5266 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.487665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3738a88-0db5-4217-907c-196fd9445672 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.584852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:34.594298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.595296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.595754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.596223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.599438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:34.601403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3738a88-0db5-4217-907c-196fd9445672 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.608501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.609208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.609770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.610250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.610968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.613376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.648720 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1166/2305] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:34 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:34.648720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.649018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:34.672676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd25e50-47ba-4852-95bc-5915f0e8f3c9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.681902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.683171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.683459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.686655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] PATH_INFO: `/v3/projects/4a0e8c1869b647449387361c894de25a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.689115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0e8c1869b647449387361c894de25a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.691763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.713318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:34.716381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.721196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.736373 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1140/2306] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:34.737340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.758454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca60c6e-6b3e-4a19-905c-5ec41e89a380 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.796613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0e8c1869b647449387361c894de25a', 'request_id': 'req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:34.806816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0e8c1869b647449387361c894de25a', 'request_id': 'req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:34.814976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0e8c1869b647449387361c894de25a', 'request_id': 'req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:34.844707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] The token cache is being invalidate because project 4a0e8c1869b647449387361c894de25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:34.844995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:34.845770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:34.848109 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1167/2307] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:34 2022] DELETE /identity/v3/projects/4a0e8c1869b647449387361c894de25a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:34.852770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.855114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:34.871186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.875014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.875303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.875632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.876378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.878050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.892230 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1141/2308] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:34.897967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:34.900179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:34.916763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61214cb-a71b-4b93-9ded-6496bddbe954 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.923288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd528ee2-067d-4f9f-826b-5a211f9d844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.924153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd528ee2-067d-4f9f-826b-5a211f9d844d admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.924441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd528ee2-067d-4f9f-826b-5a211f9d844d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.925165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd528ee2-067d-4f9f-826b-5a211f9d844d admin admin] PATH_INFO: `/v3/projects/ff0163b81ff044beba8e16871c4f5f3b/users/2d39f05541cf45eca3052140d7927286/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.951794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ea5e1-5996-4b4c-8ea7-548cb8e960c8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:34.953799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd528ee2-067d-4f9f-826b-5a211f9d844d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0163b81ff044beba8e16871c4f5f3b, user_id=2d39f05541cf45eca3052140d7927286, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.956632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd528ee2-067d-4f9f-826b-5a211f9d844d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.956831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:34.957469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:34.957785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:34.958214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] PATH_INFO: `/v3/users/e349e6a72df046ad8e5e3d43e0bd7dd2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:34.959496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e349e6a72df046ad8e5e3d43e0bd7dd2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:34.961413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:34.971782 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1142/2309] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:34 2022] PUT /identity/v3/projects/ff0163b81ff044beba8e16871c4f5f3b/users/2d39f05541cf45eca3052140d7927286/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:34.976418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd528ee2-067d-4f9f-826b-5a211f9d844d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:35.019616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e349e6a72df046ad8e5e3d43e0bd7dd2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.027366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e349e6a72df046ad8e5e3d43e0bd7dd2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.028381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e349e6a72df046ad8e5e3d43e0bd7dd2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.029201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e349e6a72df046ad8e5e3d43e0bd7dd2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.040237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e349e6a72df046ad8e5e3d43e0bd7dd2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.052873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd528ee2-067d-4f9f-826b-5a211f9d844d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:35.059051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095239a4-da0c-491a-a7ec-07a6f58b932e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:35.059698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095239a4-da0c-491a-a7ec-07a6f58b932e admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:35.059977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095239a4-da0c-491a-a7ec-07a6f58b932e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:35.060247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095239a4-da0c-491a-a7ec-07a6f58b932e admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:35.061056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095239a4-da0c-491a-a7ec-07a6f58b932e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:35.063153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095239a4-da0c-491a-a7ec-07a6f58b932e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:35.069465 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1168/2310] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:34 2022] DELETE /identity/v3/users/e349e6a72df046ad8e5e3d43e0bd7dd2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:35.081709 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1143/2311] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:35.083080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:35.087571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:35.125887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:35.181664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cf680-fdb6-4577-b7ae-6e80698a4371 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:35.188644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:35.189389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:35.189536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:35.190108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] PATH_INFO: `/v3/projects/ff0163b81ff044beba8e16871c4f5f3b/users/2d39f05541cf45eca3052140d7927286/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:35.192915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0163b81ff044beba8e16871c4f5f3b, user_id=2d39f05541cf45eca3052140d7927286, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:35.197446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:35.221716 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1169/2312] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:35 2022] PUT /identity/v3/projects/ff0163b81ff044beba8e16871c4f5f3b/users/2d39f05541cf45eca3052140d7927286/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:35.224818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744abe7-3e6d-48c6-bd1f-aac67ec2d4a1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:35.225114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744abe7-3e6d-48c6-bd1f-aac67ec2d4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:35.225415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744abe7-3e6d-48c6-bd1f-aac67ec2d4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:35.233500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:35.265422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1744abe7-3e6d-48c6-bd1f-aac67ec2d4a1 None None] MFA Rules not processed for user `2d39f05541cf45eca3052140d7927286`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:35.266800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1744abe7-3e6d-48c6-bd1f-aac67ec2d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:35.353243 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1144/2313] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:35.365408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:35.367132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:35.429825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744abe7-3e6d-48c6-bd1f-aac67ec2d4a1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:35.431740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1744abe7-3e6d-48c6-bd1f-aac67ec2d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:35.442556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ee6cb-036d-4704-bca2-15ebcd212fb1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:35.447795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:35.448224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:35.448582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:35.449444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:35.451798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:35.514412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744abe7-3e6d-48c6-bd1f-aac67ec2d4a1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:35.515814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:35.517262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:35.519614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:35.520842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:35.521123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:35.521399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] PATH_INFO: `/v3/projects/da0fa7fbdb93488dbb1c151991aa98f4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:35.522720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da0fa7fbdb93488dbb1c151991aa98f4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:35.524645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:35.527222 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1170/2314] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:35.575576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da0fa7fbdb93488dbb1c151991aa98f4', 'request_id': 'req-0167a3a4-a215-4ae1-aa89-d7e269bfb777'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.583764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da0fa7fbdb93488dbb1c151991aa98f4', 'request_id': 'req-0167a3a4-a215-4ae1-aa89-d7e269bfb777'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.590542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da0fa7fbdb93488dbb1c151991aa98f4', 'request_id': 'req-0167a3a4-a215-4ae1-aa89-d7e269bfb777'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.602408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] The token cache is being invalidate because project da0fa7fbdb93488dbb1c151991aa98f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:35.602706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.603377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.604878 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1145/2315] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:35 2022] DELETE /identity/v3/projects/da0fa7fbdb93488dbb1c151991aa98f4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:35.608991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:35.620584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ae9a4-0ccb-45f0-85de-1d9bb014c8ab service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:35.678959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:35.679698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:35.680055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:35.680408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] PATH_INFO: `/v3/users/1598b9d5133849128867541fbc8097e3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:35.681792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1598b9d5133849128867541fbc8097e3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:35.684214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:35.698453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:35.737469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1598b9d5133849128867541fbc8097e3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.745422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1598b9d5133849128867541fbc8097e3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.746169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1598b9d5133849128867541fbc8097e3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.746876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1598b9d5133849128867541fbc8097e3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.754693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e6990abe-9afd-480b-9997-d8556f081289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1598b9d5133849128867541fbc8097e3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:35.774447 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1171/2316] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:35 2022] DELETE /identity/v3/users/1598b9d5133849128867541fbc8097e3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:36.676943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:36.678417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:36.691484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167a3a4-a215-4ae1-aa89-d7e269bfb777 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:36.708306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:36.760761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:36.761535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:36.761877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:36.762154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] PATH_INFO: `/v3/projects/113787f3f77d4c7d81df50abfdba6121` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:36.763310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=113787f3f77d4c7d81df50abfdba6121)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:36.765254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:36.815039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '113787f3f77d4c7d81df50abfdba6121', 'request_id': 'req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:36.823442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '113787f3f77d4c7d81df50abfdba6121', 'request_id': 'req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:36.829965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '113787f3f77d4c7d81df50abfdba6121', 'request_id': 'req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:36.841092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] The token cache is being invalidate because project 113787f3f77d4c7d81df50abfdba6121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:36.841389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:36.842069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2d2af8e7-be4d-4a99-bba6-a8ca297d6799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:36.843568 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1146/2317] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:36 2022] DELETE /identity/v3/projects/113787f3f77d4c7d81df50abfdba6121 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:36.848961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808bfef-a56a-4485-b620-cee5ee4072ba None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:36.849461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808bfef-a56a-4485-b620-cee5ee4072ba None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:36.849554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808bfef-a56a-4485-b620-cee5ee4072ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:36.872717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a808bfef-a56a-4485-b620-cee5ee4072ba None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:36.874224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a808bfef-a56a-4485-b620-cee5ee4072ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:36.956984 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1172/2318] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:36.969369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67950-a34e-41df-95c6-625e71eeb324 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:36.969369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67950-a34e-41df-95c6-625e71eeb324 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:36.969369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67950-a34e-41df-95c6-625e71eeb324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:36.994131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-67e67950-a34e-41df-95c6-625e71eeb324 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:36.995400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-67e67950-a34e-41df-95c6-625e71eeb324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:37.026263 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1147/2319] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:37.031056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808bfef-a56a-4485-b620-cee5ee4072ba None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:37.061496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808bfef-a56a-4485-b620-cee5ee4072ba None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:37.067942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:37.068787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.069274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.069809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.070706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:37.071231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:37.073132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:37.096269 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1173/2320] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:37.103295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e67950-a34e-41df-95c6-625e71eeb324 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:37.113937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e67950-a34e-41df-95c6-625e71eeb324 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:37.119836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b06f6-b286-470d-9c0e-9d321ac0fa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:37.120541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b06f6-b286-470d-9c0e-9d321ac0fa69 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.120919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b06f6-b286-470d-9c0e-9d321ac0fa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.121286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b06f6-b286-470d-9c0e-9d321ac0fa69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.122236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b06f6-b286-470d-9c0e-9d321ac0fa69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:37.124243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b06f6-b286-470d-9c0e-9d321ac0fa69 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:37.145601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:37.152453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:37.184470 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1148/2321] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:37 2022] POST /identity/v3/projects => generated 363 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:37.189592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b06f6-b286-470d-9c0e-9d321ac0fa69 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:37.267167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e84b32-93eb-46e4-ae9b-bfbaee858267 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:37.314630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:37.315187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b06f6-b286-470d-9c0e-9d321ac0fa69 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:37.333682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25e098-c18e-439c-a0c3-acd088d785f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:37.334293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25e098-c18e-439c-a0c3-acd088d785f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.334575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25e098-c18e-439c-a0c3-acd088d785f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.334848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25e098-c18e-439c-a0c3-acd088d785f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.335646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25e098-c18e-439c-a0c3-acd088d785f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:37.338078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25e098-c18e-439c-a0c3-acd088d785f8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:37.341401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:37.342230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.342675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.343092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.345443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:37.387271 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1149/2322] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:37 2022] POST /identity/v3/users => generated 282 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:37.388542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25e098-c18e-439c-a0c3-acd088d785f8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:37.415485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25e098-c18e-439c-a0c3-acd088d785f8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:37.423600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f5833-a854-493a-bdb0-8571b5d1f97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:37.424483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f5833-a854-493a-bdb0-8571b5d1f97d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.424897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f5833-a854-493a-bdb0-8571b5d1f97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.425930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f5833-a854-493a-bdb0-8571b5d1f97d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.426674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f5833-a854-493a-bdb0-8571b5d1f97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:37.430307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f5833-a854-493a-bdb0-8571b5d1f97d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:37.434121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:37.462158 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1150/2323] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:37.466441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f5833-a854-493a-bdb0-8571b5d1f97d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:37.483875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:37.486354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:37.487531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f5833-a854-493a-bdb0-8571b5d1f97d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:37.507895 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1174/2324] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:37.510834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bfbfc-1c03-4d62-ad95-fa8aad950dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:37.511575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bfbfc-1c03-4d62-ad95-fa8aad950dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.512318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bfbfc-1c03-4d62-ad95-fa8aad950dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.512318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bfbfc-1c03-4d62-ad95-fa8aad950dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.513065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bfbfc-1c03-4d62-ad95-fa8aad950dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:37.515040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bfbfc-1c03-4d62-ad95-fa8aad950dd5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:37.536155 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1151/2325] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:37.541021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:37.553743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b52284-5055-403c-a214-5df148a195e7 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:37.561139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:37.561851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.562122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.562395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] PATH_INFO: `/v3/projects/51c74da8d75b4421a3ac2440584f1551/users/7b71b56279a94ad5bde3fb2a66bde7a8/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.597958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c74da8d75b4421a3ac2440584f1551, user_id=7b71b56279a94ad5bde3fb2a66bde7a8, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:37.597958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:37.615277 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1175/2326] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:37 2022] PUT /identity/v3/projects/51c74da8d75b4421a3ac2440584f1551/users/7b71b56279a94ad5bde3fb2a66bde7a8/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:37.626438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f006f7-82c7-44c9-a8c6-7d206a1a9b69 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.626936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f006f7-82c7-44c9-a8c6-7d206a1a9b69 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.627693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f006f7-82c7-44c9-a8c6-7d206a1a9b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.670015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-01f006f7-82c7-44c9-a8c6-7d206a1a9b69 None None] MFA Rules not processed for user `7b71b56279a94ad5bde3fb2a66bde7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:37.674444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-01f006f7-82c7-44c9-a8c6-7d206a1a9b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:37.777191 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1152/2327] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:37.865117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:37.867157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:37.950223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f939f-1cab-4350-8797-1d5e32ff266d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:37.953698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:37.954301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:37.954577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:37.954890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:37.970970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:37.973076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:37.983137 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1176/2328] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:38.537789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f006f7-82c7-44c9-a8c6-7d206a1a9b69 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:38.555745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f006f7-82c7-44c9-a8c6-7d206a1a9b69 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:38.562003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca343aa9-9877-49b3-9589-8a17701b1cd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:38.562908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343aa9-9877-49b3-9589-8a17701b1cd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:38.563420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343aa9-9877-49b3-9589-8a17701b1cd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:38.563944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343aa9-9877-49b3-9589-8a17701b1cd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:38.574896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ca343aa9-9877-49b3-9589-8a17701b1cd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:38.693439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca343aa9-9877-49b3-9589-8a17701b1cd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:38.696217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca343aa9-9877-49b3-9589-8a17701b1cd4 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:38.721781 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1153/2329] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:39.858513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:39.860498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:39.983615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18ecb3-d283-4b94-b451-a5389c3651c2 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:39.995637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:39.996286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:39.996567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:39.996841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] PATH_INFO: `/v3/users/dca4915b52e043aea7debb5ddde9d4a6` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:39.998022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca4915b52e043aea7debb5ddde9d4a6)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:40.000244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:40.007816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:40.094404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca4915b52e043aea7debb5ddde9d4a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.125782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca4915b52e043aea7debb5ddde9d4a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.125782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca4915b52e043aea7debb5ddde9d4a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.125782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca4915b52e043aea7debb5ddde9d4a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.127600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca4915b52e043aea7debb5ddde9d4a6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.155139 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1177/2330] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:39 2022] DELETE /identity/v3/users/dca4915b52e043aea7debb5ddde9d4a6 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:40.161545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acbc92-daa3-4277-9333-fd66a2bc1866 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:40.163144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acbc92-daa3-4277-9333-fd66a2bc1866 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:40.163440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acbc92-daa3-4277-9333-fd66a2bc1866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:40.190434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-09acbc92-daa3-4277-9333-fd66a2bc1866 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:40.191739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-09acbc92-daa3-4277-9333-fd66a2bc1866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:40.281781 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1154/2331] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:40.291213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:40.292787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:40.365089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cfbd4-29e4-493a-b1f6-8b0c89380701 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:40.369106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa58195-5322-4717-8f36-a7de41669c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:40.370036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa58195-5322-4717-8f36-a7de41669c3e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:40.370428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa58195-5322-4717-8f36-a7de41669c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:40.370808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa58195-5322-4717-8f36-a7de41669c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:40.381410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa58195-5322-4717-8f36-a7de41669c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:40.382896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa58195-5322-4717-8f36-a7de41669c3e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:40.394668 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1178/2332] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:40.691547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acbc92-daa3-4277-9333-fd66a2bc1866 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:40.693028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-09acbc92-daa3-4277-9333-fd66a2bc1866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:40.705266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acbc92-daa3-4277-9333-fd66a2bc1866 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:40.710061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:40.710699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:40.710978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:40.711252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] PATH_INFO: `/v3/projects/6f2c247f903a4f688450629a5fc7d6ca` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:40.712383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2c247f903a4f688450629a5fc7d6ca)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:40.714335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:40.721056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:40.765965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2c247f903a4f688450629a5fc7d6ca', 'request_id': 'req-f2b8a33f-234f-4192-b6c4-6847a0ac9346'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.774322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2c247f903a4f688450629a5fc7d6ca', 'request_id': 'req-f2b8a33f-234f-4192-b6c4-6847a0ac9346'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.781113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2c247f903a4f688450629a5fc7d6ca', 'request_id': 'req-f2b8a33f-234f-4192-b6c4-6847a0ac9346'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.792282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] The token cache is being invalidate because project 6f2c247f903a4f688450629a5fc7d6ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:40.792572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.793216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.794924 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1155/2333] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:40 2022] DELETE /identity/v3/projects/6f2c247f903a4f688450629a5fc7d6ca => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:40.798661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa58195-5322-4717-8f36-a7de41669c3e service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:40.810610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa58195-5322-4717-8f36-a7de41669c3e service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:40.874754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:40.875441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:40.875649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:40.875919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] PATH_INFO: `/v3/users/0f0816c367aa42d19ce577c1f8ce5b49` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:40.877078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f0816c367aa42d19ce577c1f8ce5b49)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:40.881120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:40.916913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:40.933860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f0816c367aa42d19ce577c1f8ce5b49'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.941441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f0816c367aa42d19ce577c1f8ce5b49'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.942705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f0816c367aa42d19ce577c1f8ce5b49'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.943337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f0816c367aa42d19ce577c1f8ce5b49'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.950690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f0816c367aa42d19ce577c1f8ce5b49'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:40.977319 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1179/2334] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:40 2022] DELETE /identity/v3/users/0f0816c367aa42d19ce577c1f8ce5b49 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:41.350883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:41.356443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:41.393850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8a33f-234f-4192-b6c4-6847a0ac9346 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:41.469823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:41.470503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:41.470780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:41.471263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] PATH_INFO: `/v3/projects/23e6a3dd20c7484b8946bb73bbdd7b2c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:41.472712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e6a3dd20c7484b8946bb73bbdd7b2c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:41.474654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:41.537838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e6a3dd20c7484b8946bb73bbdd7b2c', 'request_id': 'req-cbe987d5-4aff-41eb-a194-374d773e1ddb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.546449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e6a3dd20c7484b8946bb73bbdd7b2c', 'request_id': 'req-cbe987d5-4aff-41eb-a194-374d773e1ddb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.555240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e6a3dd20c7484b8946bb73bbdd7b2c', 'request_id': 'req-cbe987d5-4aff-41eb-a194-374d773e1ddb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.569978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] The token cache is being invalidate because project 23e6a3dd20c7484b8946bb73bbdd7b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:41.570199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.570857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.572403 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1156/2335] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:41 2022] DELETE /identity/v3/projects/23e6a3dd20c7484b8946bb73bbdd7b2c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:41.576563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:41.578062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:41.592470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b322-ba89-412b-96b2-c7fc1fe62543 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:41.650929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:41.651572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:41.651852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:41.652124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] PATH_INFO: `/v3/users/51998ac1d5d24c069ef23f61d778ac53` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:41.653543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51998ac1d5d24c069ef23f61d778ac53)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:41.655593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:41.706032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51998ac1d5d24c069ef23f61d778ac53'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.713676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51998ac1d5d24c069ef23f61d778ac53'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.714347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51998ac1d5d24c069ef23f61d778ac53'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.714981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51998ac1d5d24c069ef23f61d778ac53'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.722135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3f41bef3-bbb3-495d-b359-b1783a061fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51998ac1d5d24c069ef23f61d778ac53'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:41.741598 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1180/2336] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:41 2022] DELETE /identity/v3/users/51998ac1d5d24c069ef23f61d778ac53 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:42.008565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:42.010424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:42.030625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe987d5-4aff-41eb-a194-374d773e1ddb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:42.051938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:42.090654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:42.091222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.091561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.091780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] PATH_INFO: `/v3/projects/fc82cf17bf52476fa03ff9cb83713361` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.092907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc82cf17bf52476fa03ff9cb83713361)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:42.094864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:42.139926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc82cf17bf52476fa03ff9cb83713361', 'request_id': 'req-243e35b3-e950-4790-839d-b6ee3b1836aa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:42.147944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc82cf17bf52476fa03ff9cb83713361', 'request_id': 'req-243e35b3-e950-4790-839d-b6ee3b1836aa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:42.154052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc82cf17bf52476fa03ff9cb83713361', 'request_id': 'req-243e35b3-e950-4790-839d-b6ee3b1836aa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:42.165127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] The token cache is being invalidate because project fc82cf17bf52476fa03ff9cb83713361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:42.165412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:42.166067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-243e35b3-e950-4790-839d-b6ee3b1836aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:42.167523 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1157/2337] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:42 2022] DELETE /identity/v3/projects/fc82cf17bf52476fa03ff9cb83713361 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:42.172470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd0837-609b-44b3-937a-8f3adbbe0bef None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.172981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd0837-609b-44b3-937a-8f3adbbe0bef None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.173104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd0837-609b-44b3-937a-8f3adbbe0bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.192576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d8fd0837-609b-44b3-937a-8f3adbbe0bef None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:42.193816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d8fd0837-609b-44b3-937a-8f3adbbe0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:42.214593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:42.255788 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1181/2338] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:42.263562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70131f3c-6862-4e37-9e25-f4a0581ca327 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.263844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70131f3c-6862-4e37-9e25-f4a0581ca327 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.264117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70131f3c-6862-4e37-9e25-f4a0581ca327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.285565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-70131f3c-6862-4e37-9e25-f4a0581ca327 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:42.286832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-70131f3c-6862-4e37-9e25-f4a0581ca327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:42.295046 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1158/2339] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:42.299210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd0837-609b-44b3-937a-8f3adbbe0bef None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:42.309067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd0837-609b-44b3-937a-8f3adbbe0bef None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:42.313728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:42.314323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.314602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.314874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.315530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:42.315934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:42.317538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:42.328962 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1182/2340] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:42.333756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70131f3c-6862-4e37-9e25-f4a0581ca327 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:42.342968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70131f3c-6862-4e37-9e25-f4a0581ca327 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:42.348317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa16b55-67dd-49db-ac37-c1e02deb7dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:42.348924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa16b55-67dd-49db-ac37-c1e02deb7dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.349208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa16b55-67dd-49db-ac37-c1e02deb7dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.349511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa16b55-67dd-49db-ac37-c1e02deb7dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.350180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa16b55-67dd-49db-ac37-c1e02deb7dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:42.352135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa16b55-67dd-49db-ac37-c1e02deb7dc0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:42.398351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:42.403677 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1159/2341] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:42 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:42.416416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a99cdf-0b3d-4ae0-90dd-aac5431dc2ce admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:42.558775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd39b42-f9ae-4fd1-9f7c-f5655f7ac650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:42.559573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd39b42-f9ae-4fd1-9f7c-f5655f7ac650 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.559861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd39b42-f9ae-4fd1-9f7c-f5655f7ac650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.560134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd39b42-f9ae-4fd1-9f7c-f5655f7ac650 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.560770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd39b42-f9ae-4fd1-9f7c-f5655f7ac650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:42.562778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd39b42-f9ae-4fd1-9f7c-f5655f7ac650 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:42.611486 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1183/2342] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:42 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:42.614434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa16b55-67dd-49db-ac37-c1e02deb7dc0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:42.642196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa16b55-67dd-49db-ac37-c1e02deb7dc0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:42.657817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a6e5-8f06-4811-8847-f79840005a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:42.658540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a6e5-8f06-4811-8847-f79840005a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.658896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a6e5-8f06-4811-8847-f79840005a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.659273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213a6e5-8f06-4811-8847-f79840005a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.660284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213a6e5-8f06-4811-8847-f79840005a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:42.661944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213a6e5-8f06-4811-8847-f79840005a39 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:42.690204 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1160/2343] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:42.694911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd39b42-f9ae-4fd1-9f7c-f5655f7ac650 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:42.723647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd39b42-f9ae-4fd1-9f7c-f5655f7ac650 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:42.729085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba755570-e210-4529-84c3-d99514126308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:42.729840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba755570-e210-4529-84c3-d99514126308 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.730131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba755570-e210-4529-84c3-d99514126308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.730405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba755570-e210-4529-84c3-d99514126308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.731100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba755570-e210-4529-84c3-d99514126308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:42.732900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba755570-e210-4529-84c3-d99514126308 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:42.749421 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1184/2344] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:42.753809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a6e5-8f06-4811-8847-f79840005a39 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:42.764956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213a6e5-8f06-4811-8847-f79840005a39 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:42.771623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:42.772296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.772580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.772859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] PATH_INFO: `/v3/projects/93c1c087dc60448e82e206f3082e58ff/users/f1e20302069242f5b7dcd2647c977057/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.797744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c1c087dc60448e82e206f3082e58ff, user_id=f1e20302069242f5b7dcd2647c977057, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:42.802070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:42.821345 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1161/2345] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:42 2022] PUT /identity/v3/projects/93c1c087dc60448e82e206f3082e58ff/users/f1e20302069242f5b7dcd2647c977057/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:42.833844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fd3fe-be98-4982-9a05-7304fa800dfa None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:42.835796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fd3fe-be98-4982-9a05-7304fa800dfa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:42.836176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fd3fe-be98-4982-9a05-7304fa800dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:42.870339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-de6fd3fe-be98-4982-9a05-7304fa800dfa None None] MFA Rules not processed for user `f1e20302069242f5b7dcd2647c977057`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:42.871546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-de6fd3fe-be98-4982-9a05-7304fa800dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:42.955243 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1185/2346] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:42.967629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:42.969331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:43.025988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4a6fa-eec5-4984-9fe3-feaec30cbeb7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:43.029699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cd275-233c-48df-88da-2ede7ab954a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fe7a090bd7941bb827d301aa4949857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:43.030309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cd275-233c-48df-88da-2ede7ab954a8 service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:43.030591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cd275-233c-48df-88da-2ede7ab954a8 service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:43.030866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cd275-233c-48df-88da-2ede7ab954a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:43.036711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:43.041146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cd275-233c-48df-88da-2ede7ab954a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:43.043120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cd275-233c-48df-88da-2ede7ab954a8 service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:43.052658 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1162/2347] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:44.045835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fd3fe-be98-4982-9a05-7304fa800dfa None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:44.049027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-de6fd3fe-be98-4982-9a05-7304fa800dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:44.118504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:44.215342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fd3fe-be98-4982-9a05-7304fa800dfa None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:44.227551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:44.228209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:44.228488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:44.228757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] PATH_INFO: `/v3/users/75e2aec5891f4b519623680718aa1d28` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:44.229947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e2aec5891f4b519623680718aa1d28)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:44.231899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:44.291367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e2aec5891f4b519623680718aa1d28'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:44.298205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e2aec5891f4b519623680718aa1d28'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:44.298868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e2aec5891f4b519623680718aa1d28'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:44.299537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e2aec5891f4b519623680718aa1d28'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:44.312594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e2aec5891f4b519623680718aa1d28'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:44.332082 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1186/2348] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:44 2022] DELETE /identity/v3/users/75e2aec5891f4b519623680718aa1d28 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:44.336734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43c50-f871-4ccb-9703-19e4279de1ce None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:44.337241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43c50-f871-4ccb-9703-19e4279de1ce None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:44.337354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b43c50-f871-4ccb-9703-19e4279de1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:44.358463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d0b43c50-f871-4ccb-9703-19e4279de1ce None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:44.359670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d0b43c50-f871-4ccb-9703-19e4279de1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:44.424893 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1163/2349] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:44.434828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:44.436229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:44.495187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d866742-44ed-4666-b3b2-d457abbd0407 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:44.498866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164a1ce-9a21-4619-8683-4e6985f67824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:44.499479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a1ce-9a21-4619-8683-4e6985f67824 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:44.499855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a1ce-9a21-4619-8683-4e6985f67824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:44.500024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a1ce-9a21-4619-8683-4e6985f67824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:44.510979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164a1ce-9a21-4619-8683-4e6985f67824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:44.512467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164a1ce-9a21-4619-8683-4e6985f67824 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:44.524197 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1187/2350] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:44.883101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b43c50-f871-4ccb-9703-19e4279de1ce None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:44.884714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d0b43c50-f871-4ccb-9703-19e4279de1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:44.904171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b43c50-f871-4ccb-9703-19e4279de1ce None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:44.922495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:44.923738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:44.923738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:44.924589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] PATH_INFO: `/v3/projects/2a168585ae4149f58a32da9be0b1878e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:44.926549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a168585ae4149f58a32da9be0b1878e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:44.929168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:44.998946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a168585ae4149f58a32da9be0b1878e', 'request_id': 'req-9746e767-78e9-41a2-99bb-79039c549fd3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.020859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a168585ae4149f58a32da9be0b1878e', 'request_id': 'req-9746e767-78e9-41a2-99bb-79039c549fd3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.027679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a168585ae4149f58a32da9be0b1878e', 'request_id': 'req-9746e767-78e9-41a2-99bb-79039c549fd3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.038973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] The token cache is being invalidate because project 2a168585ae4149f58a32da9be0b1878e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:45.039368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.040199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.041939 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1164/2351] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:44 2022] DELETE /identity/v3/projects/2a168585ae4149f58a32da9be0b1878e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:45.046042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164a1ce-9a21-4619-8683-4e6985f67824 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:45.057823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164a1ce-9a21-4619-8683-4e6985f67824 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:45.107218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:45.124894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:45.125525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:45.125824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:45.126096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] PATH_INFO: `/v3/users/d7802e5aa89c464a9313a338a76f38bf` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:45.127208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7802e5aa89c464a9313a338a76f38bf)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:45.129811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:45.198506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7802e5aa89c464a9313a338a76f38bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.233987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7802e5aa89c464a9313a338a76f38bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.233987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7802e5aa89c464a9313a338a76f38bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.233987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7802e5aa89c464a9313a338a76f38bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.233987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7802e5aa89c464a9313a338a76f38bf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.256019 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1188/2352] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:45 2022] DELETE /identity/v3/users/d7802e5aa89c464a9313a338a76f38bf => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:45.687676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:45.689173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:45.722155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746e767-78e9-41a2-99bb-79039c549fd3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:45.818955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:45.820135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:45.820474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:45.820822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] PATH_INFO: `/v3/users/db77ee86ba0a4836990199e57f42ff49` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:45.822266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=db77ee86ba0a4836990199e57f42ff49)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:45.824808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:45.860112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:45.901840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db77ee86ba0a4836990199e57f42ff49'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.910892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db77ee86ba0a4836990199e57f42ff49'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.911875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db77ee86ba0a4836990199e57f42ff49'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.912927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db77ee86ba0a4836990199e57f42ff49'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.920995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db77ee86ba0a4836990199e57f42ff49'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:45.926813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:45.928881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:45.944776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd616bf-451c-4676-aa72-df7ddc1c4b0b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:45.949834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:45.951338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:45.951704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:45.952049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] PATH_INFO: `/v3/projects/e9a0fc8336054e42811fe6ba43ac20dd` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:45.953245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a0fc8336054e42811fe6ba43ac20dd)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:45.955627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:45.957895 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1165/2353] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:45 2022] DELETE /identity/v3/users/db77ee86ba0a4836990199e57f42ff49 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:46.012632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a0fc8336054e42811fe6ba43ac20dd', 'request_id': 'req-43c250a8-6386-4e71-9d3a-2b2ca853692d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.021392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a0fc8336054e42811fe6ba43ac20dd', 'request_id': 'req-43c250a8-6386-4e71-9d3a-2b2ca853692d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.030572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a0fc8336054e42811fe6ba43ac20dd', 'request_id': 'req-43c250a8-6386-4e71-9d3a-2b2ca853692d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.048117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] The token cache is being invalidate because project e9a0fc8336054e42811fe6ba43ac20dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:46.050739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.052859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.057973 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1189/2354] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:45 2022] DELETE /identity/v3/projects/e9a0fc8336054e42811fe6ba43ac20dd => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:46.063080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:46.066809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:46.171137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583f360-0128-4d3a-898b-01febc71b7ea admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:46.193923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:46.195716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:46.254541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:46.255358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:46.255718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:46.256046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] PATH_INFO: `/v3/users/2d39f05541cf45eca3052140d7927286` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:46.257214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d39f05541cf45eca3052140d7927286)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:46.259188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:46.279697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:46.306609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c250a8-6386-4e71-9d3a-2b2ca853692d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:46.318408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:46.319893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:46.320388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:46.326227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] PATH_INFO: `/v3/projects/7864f95a786b4b41ae1e66a2ae2acee1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:46.327678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7864f95a786b4b41ae1e66a2ae2acee1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:46.327815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:46.329738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:46.330284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d39f05541cf45eca3052140d7927286'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.340631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d39f05541cf45eca3052140d7927286'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.341366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d39f05541cf45eca3052140d7927286'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.343631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d39f05541cf45eca3052140d7927286'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.356445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d39f05541cf45eca3052140d7927286'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.382695 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1166/2355] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:46 2022] DELETE /identity/v3/users/2d39f05541cf45eca3052140d7927286 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:46.426320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7864f95a786b4b41ae1e66a2ae2acee1', 'request_id': 'req-a0846b51-d612-455a-a103-38f533f7a808'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.434732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7864f95a786b4b41ae1e66a2ae2acee1', 'request_id': 'req-a0846b51-d612-455a-a103-38f533f7a808'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.443079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7864f95a786b4b41ae1e66a2ae2acee1', 'request_id': 'req-a0846b51-d612-455a-a103-38f533f7a808'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.455678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] The token cache is being invalidate because project 7864f95a786b4b41ae1e66a2ae2acee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:46.455885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.456599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.458457 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1190/2356] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:46 2022] DELETE /identity/v3/projects/7864f95a786b4b41ae1e66a2ae2acee1 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:46.462514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:46.464270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:46.568936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6a27b-985c-488b-a1b7-a0152da2104a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:46.644941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:46.645993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:46.646270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:46.646575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] PATH_INFO: `/v3/users/1ad87a658f3e41f9b0acb3c2785207b4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:46.647720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad87a658f3e41f9b0acb3c2785207b4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:46.649666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:46.703273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad87a658f3e41f9b0acb3c2785207b4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.710929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad87a658f3e41f9b0acb3c2785207b4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.714957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad87a658f3e41f9b0acb3c2785207b4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.716147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad87a658f3e41f9b0acb3c2785207b4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.728146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad87a658f3e41f9b0acb3c2785207b4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.740105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:46.753602 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1167/2357] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:46 2022] DELETE /identity/v3/users/1ad87a658f3e41f9b0acb3c2785207b4 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:46.773286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:46.774798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:46.791955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0846b51-d612-455a-a103-38f533f7a808 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:46.879927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:46.880691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:46.881074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:46.881449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] PATH_INFO: `/v3/projects/ff0163b81ff044beba8e16871c4f5f3b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:46.882867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0163b81ff044beba8e16871c4f5f3b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:46.885116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:46.951308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0163b81ff044beba8e16871c4f5f3b', 'request_id': 'req-0e146892-3098-4999-a594-cfb7786a712e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.964343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0163b81ff044beba8e16871c4f5f3b', 'request_id': 'req-0e146892-3098-4999-a594-cfb7786a712e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.981678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0163b81ff044beba8e16871c4f5f3b', 'request_id': 'req-0e146892-3098-4999-a594-cfb7786a712e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:46.997943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:46.999707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:47.000022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] The token cache is being invalidate because project ff0163b81ff044beba8e16871c4f5f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:47.000325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:47.000990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e146892-3098-4999-a594-cfb7786a712e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:47.003240 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1191/2358] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:46 2022] DELETE /identity/v3/projects/ff0163b81ff044beba8e16871c4f5f3b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:47.012098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7be42-f1d2-403f-8d48-7468744bc5d6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.012596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7be42-f1d2-403f-8d48-7468744bc5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.012948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7be42-f1d2-403f-8d48-7468744bc5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.029829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:47.040512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dac7be42-f1d2-403f-8d48-7468744bc5d6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:47.042040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dac7be42-f1d2-403f-8d48-7468744bc5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:47.049216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ac34-4aa8-4fd1-97dd-248e3fd5b36c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.165194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.166064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.166625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.167067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] PATH_INFO: `/v3/projects/42428ee5681f40718b9a855ec5d61533` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.168222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42428ee5681f40718b9a855ec5d61533)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.181368 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1192/2359] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:47.182019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.182142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69b914-a22d-4d66-b601-7bea75239f3e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.182142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69b914-a22d-4d66-b601-7bea75239f3e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.182142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69b914-a22d-4d66-b601-7bea75239f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.204549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1a69b914-a22d-4d66-b601-7bea75239f3e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:47.206849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1a69b914-a22d-4d66-b601-7bea75239f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:47.222687 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1193/2360] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:47.226983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69b914-a22d-4d66-b601-7bea75239f3e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.236531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69b914-a22d-4d66-b601-7bea75239f3e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.249767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42428ee5681f40718b9a855ec5d61533', 'request_id': 'req-550e2ac6-f668-4ac5-a260-152497281d83'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:47.261774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42428ee5681f40718b9a855ec5d61533', 'request_id': 'req-550e2ac6-f668-4ac5-a260-152497281d83'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:47.272992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42428ee5681f40718b9a855ec5d61533', 'request_id': 'req-550e2ac6-f668-4ac5-a260-152497281d83'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:47.290927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] The token cache is being invalidate because project 42428ee5681f40718b9a855ec5d61533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:47.291480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:47.292633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-550e2ac6-f668-4ac5-a260-152497281d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:47.295846 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1168/2361] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:46 2022] DELETE /identity/v3/projects/42428ee5681f40718b9a855ec5d61533 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:47.303283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e3769-83ae-433e-a0c3-5913da47361b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.303571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e3769-83ae-433e-a0c3-5913da47361b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.304366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e3769-83ae-433e-a0c3-5913da47361b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.322589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.324029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.324309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.324583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.325380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:47.325712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.327478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.340433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6e3e3769-83ae-433e-a0c3-5913da47361b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:47.341908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6e3e3769-83ae-433e-a0c3-5913da47361b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:47.342380 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1194/2362] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:37:47.348524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.350426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:47.356131 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1169/2363] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:47.365136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc43333-6489-4caa-a60a-9690a5fa9e01 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.366059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc43333-6489-4caa-a60a-9690a5fa9e01 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.366383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc43333-6489-4caa-a60a-9690a5fa9e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.367962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f773326-8b75-4cf8-a4f2-cf3dbf81e16f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.373998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50698-fa03-4a8e-820b-975eded2fb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.374490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50698-fa03-4a8e-820b-975eded2fb50 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.374751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50698-fa03-4a8e-820b-975eded2fb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.375025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50698-fa03-4a8e-820b-975eded2fb50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.375649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d50698-fa03-4a8e-820b-975eded2fb50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.377662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d50698-fa03-4a8e-820b-975eded2fb50 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.418558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3cc43333-6489-4caa-a60a-9690a5fa9e01 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:47.420012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3cc43333-6489-4caa-a60a-9690a5fa9e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:47.431170 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1170/2364] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:47.433803 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1195/2365] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:47.436061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50698-fa03-4a8e-820b-975eded2fb50 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.438523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc43333-6489-4caa-a60a-9690a5fa9e01 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.451672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc43333-6489-4caa-a60a-9690a5fa9e01 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.573992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601fdb5-ab1c-4fcb-aa04-183f600263d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.574640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fdb5-ab1c-4fcb-aa04-183f600263d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.575505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fdb5-ab1c-4fcb-aa04-183f600263d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.575795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fdb5-ab1c-4fcb-aa04-183f600263d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.576452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601fdb5-ab1c-4fcb-aa04-183f600263d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.579928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601fdb5-ab1c-4fcb-aa04-183f600263d3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.582521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d50698-fa03-4a8e-820b-975eded2fb50 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.598689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.600145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.600577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.600980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.601974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:47.602562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.604513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.620704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:47.637650 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1196/2366] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:37:47.642222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.652382 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1171/2367] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:47.653298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601fdb5-ab1c-4fcb-aa04-183f600263d3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.660222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679787eb-e9d5-4a42-bf89-dda73bd1cb87 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.667583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601fdb5-ab1c-4fcb-aa04-183f600263d3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.672472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd09e6-f066-43f8-b32f-6d8e4d1e0782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.673083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd09e6-f066-43f8-b32f-6d8e4d1e0782 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.673362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd09e6-f066-43f8-b32f-6d8e4d1e0782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.673667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd09e6-f066-43f8-b32f-6d8e4d1e0782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.674166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebe7dc-37ba-4093-9dac-b67ce7b48582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.674291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd09e6-f066-43f8-b32f-6d8e4d1e0782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.675070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebe7dc-37ba-4093-9dac-b67ce7b48582 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.675434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebe7dc-37ba-4093-9dac-b67ce7b48582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.675822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebe7dc-37ba-4093-9dac-b67ce7b48582 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.675936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd09e6-f066-43f8-b32f-6d8e4d1e0782 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.677611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebe7dc-37ba-4093-9dac-b67ce7b48582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.680367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ebe7dc-37ba-4093-9dac-b67ce7b48582 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.689541 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1172/2368] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:47.690993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:47.691257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd09e6-f066-43f8-b32f-6d8e4d1e0782 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.702661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd09e6-f066-43f8-b32f-6d8e4d1e0782 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.708255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf0304-c462-4f2b-ab25-5635a2d5e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.709118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf0304-c462-4f2b-ab25-5635a2d5e2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.709443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf0304-c462-4f2b-ab25-5635a2d5e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.709762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf0304-c462-4f2b-ab25-5635a2d5e2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.710415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf0304-c462-4f2b-ab25-5635a2d5e2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.712150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf0304-c462-4f2b-ab25-5635a2d5e2f1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.724118 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1173/2369] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:47.727867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf0304-c462-4f2b-ab25-5635a2d5e2f1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.731731 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1197/2370] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:47.735781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebe7dc-37ba-4093-9dac-b67ce7b48582 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.739905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf0304-c462-4f2b-ab25-5635a2d5e2f1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.820218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e686f-484e-41ca-932a-8b6f9a4b5386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.820874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e686f-484e-41ca-932a-8b6f9a4b5386 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.821219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e686f-484e-41ca-932a-8b6f9a4b5386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.821735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e686f-484e-41ca-932a-8b6f9a4b5386 admin admin] PATH_INFO: `/v3/projects/058c4b19eabb472ba8f41a63c65cbac0/users/bf21138ebb914753833046e748fb9a2b/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.839858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebe7dc-37ba-4093-9dac-b67ce7b48582 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.840311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e686f-484e-41ca-932a-8b6f9a4b5386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058c4b19eabb472ba8f41a63c65cbac0, user_id=bf21138ebb914753833046e748fb9a2b, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.845311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e686f-484e-41ca-932a-8b6f9a4b5386 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.848226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52e835-f4fb-4760-bd2f-c0f93e1c6412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.848996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e835-f4fb-4760-bd2f-c0f93e1c6412 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.849521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e835-f4fb-4760-bd2f-c0f93e1c6412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.849892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e835-f4fb-4760-bd2f-c0f93e1c6412 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.850821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52e835-f4fb-4760-bd2f-c0f93e1c6412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.852938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52e835-f4fb-4760-bd2f-c0f93e1c6412 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.883048 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1198/2371] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:47.884300 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1174/2372] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:47 2022] PUT /identity/v3/projects/058c4b19eabb472ba8f41a63c65cbac0/users/bf21138ebb914753833046e748fb9a2b/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:47.886837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52e835-f4fb-4760-bd2f-c0f93e1c6412 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.895485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2c76f-4e63-4139-adf5-f9e2b4c31545 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.895967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2c76f-4e63-4139-adf5-f9e2b4c31545 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.896465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2c76f-4e63-4139-adf5-f9e2b4c31545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.897329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52e835-f4fb-4760-bd2f-c0f93e1c6412 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.925961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-38f2c76f-4e63-4139-adf5-f9e2b4c31545 None None] MFA Rules not processed for user `bf21138ebb914753833046e748fb9a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:47.927589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-38f2c76f-4e63-4139-adf5-f9e2b4c31545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:47.956011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900440d-e71d-4109-a82d-0b0ae37866c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.957057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900440d-e71d-4109-a82d-0b0ae37866c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.957507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900440d-e71d-4109-a82d-0b0ae37866c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.957815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900440d-e71d-4109-a82d-0b0ae37866c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.958471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900440d-e71d-4109-a82d-0b0ae37866c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.960079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900440d-e71d-4109-a82d-0b0ae37866c7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:47.970880 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1199/2373] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:47.975434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900440d-e71d-4109-a82d-0b0ae37866c7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:47.984954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900440d-e71d-4109-a82d-0b0ae37866c7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:47.989786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bfed-791a-444e-ab2a-8d7bf8b5e79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:47.990368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bfed-791a-444e-ab2a-8d7bf8b5e79a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:47.990643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bfed-791a-444e-ab2a-8d7bf8b5e79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:47.990908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bfed-791a-444e-ab2a-8d7bf8b5e79a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:47.991528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32bfed-791a-444e-ab2a-8d7bf8b5e79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:47.991859 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1175/2374] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:37:47.993249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32bfed-791a-444e-ab2a-8d7bf8b5e79a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.007241 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1200/2375] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:48.007465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2c76f-4e63-4139-adf5-f9e2b4c31545 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.008501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-38f2c76f-4e63-4139-adf5-f9e2b4c31545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:48.008927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bfed-791a-444e-ab2a-8d7bf8b5e79a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.019319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bfed-791a-444e-ab2a-8d7bf8b5e79a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.024172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf3a5f-a4d4-4de8-a5c1-807dde3297e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.024929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf3a5f-a4d4-4de8-a5c1-807dde3297e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.025209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf3a5f-a4d4-4de8-a5c1-807dde3297e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.025478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf3a5f-a4d4-4de8-a5c1-807dde3297e4 admin admin] PATH_INFO: `/v3/projects/7550f3c2ad3d4123a750d76fbe7ae9a3/users/095cf66d04004103801fb93676892003/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.040294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf3a5f-a4d4-4de8-a5c1-807dde3297e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7550f3c2ad3d4123a750d76fbe7ae9a3, user_id=095cf66d04004103801fb93676892003, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.042972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf3a5f-a4d4-4de8-a5c1-807dde3297e4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.056075 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1201/2376] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:48 2022] PUT /identity/v3/projects/7550f3c2ad3d4123a750d76fbe7ae9a3/users/095cf66d04004103801fb93676892003/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:48.065541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:48.067993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0229f8-0cae-4d5d-847e-7d3a436d49d8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.067993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0229f8-0cae-4d5d-847e-7d3a436d49d8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.067993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0229f8-0cae-4d5d-847e-7d3a436d49d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.072225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2c76f-4e63-4139-adf5-f9e2b4c31545 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.100602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dd0229f8-0cae-4d5d-847e-7d3a436d49d8 None None] MFA Rules not processed for user `095cf66d04004103801fb93676892003`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:48.101867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dd0229f8-0cae-4d5d-847e-7d3a436d49d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:48.129182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:48.134871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da65b7-4708-4bbc-8126-da18ac8c7aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.135622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da65b7-4708-4bbc-8126-da18ac8c7aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.135799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da65b7-4708-4bbc-8126-da18ac8c7aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.136075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da65b7-4708-4bbc-8126-da18ac8c7aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.171101 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1202/2377] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:48.176583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0229f8-0cae-4d5d-847e-7d3a436d49d8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.200017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da65b7-4708-4bbc-8126-da18ac8c7aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.201461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da65b7-4708-4bbc-8126-da18ac8c7aa2 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.211364 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1176/2378] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:48.239530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da65b7-4708-4bbc-8126-da18ac8c7aa2 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.248612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0229f8-0cae-4d5d-847e-7d3a436d49d8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.258892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da65b7-4708-4bbc-8126-da18ac8c7aa2 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.261705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e10c7-4152-4db8-a6d9-c364b384f4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.262569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e10c7-4152-4db8-a6d9-c364b384f4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.263076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e10c7-4152-4db8-a6d9-c364b384f4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.263411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e10c7-4152-4db8-a6d9-c364b384f4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.264032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.264315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e10c7-4152-4db8-a6d9-c364b384f4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.265327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.265740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.266025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] PATH_INFO: `/v3/users/bf21138ebb914753833046e748fb9a2b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.266578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e10c7-4152-4db8-a6d9-c364b384f4d3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.267197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf21138ebb914753833046e748fb9a2b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.270091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.300907 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1203/2379] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:48 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:48.305039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e10c7-4152-4db8-a6d9-c364b384f4d3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.326195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf21138ebb914753833046e748fb9a2b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:48.336913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf21138ebb914753833046e748fb9a2b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:48.338536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf21138ebb914753833046e748fb9a2b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:48.339547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf21138ebb914753833046e748fb9a2b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:48.350637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b3e74de-cc70-4486-bb99-8200ddd7d2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf21138ebb914753833046e748fb9a2b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:48.378601 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1177/2380] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:48 2022] DELETE /identity/v3/users/bf21138ebb914753833046e748fb9a2b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:48.383189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e702aa-8750-4096-81da-669a8701d6ae None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.383572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e702aa-8750-4096-81da-669a8701d6ae None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.383922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e702aa-8750-4096-81da-669a8701d6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.386213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e10c7-4152-4db8-a6d9-c364b384f4d3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.416061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a6e702aa-8750-4096-81da-669a8701d6ae None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:48.417287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a6e702aa-8750-4096-81da-669a8701d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:48.444069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:48.463010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:48.471038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.471666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.472403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.472403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.472830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.474852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.487167 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1178/2381] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:48.496626 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1204/2382] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:48 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:37:48.497656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e702aa-8750-4096-81da-669a8701d6ae None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.498999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a6e702aa-8750-4096-81da-669a8701d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:48.500766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.503518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:48.519402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720deac0-a8b9-4467-a413-babed3e6471b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.525379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c02a67-99f6-4404-bee4-fde073ba2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.526035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c02a67-99f6-4404-bee4-fde073ba2e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.526319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c02a67-99f6-4404-bee4-fde073ba2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.526598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c02a67-99f6-4404-bee4-fde073ba2e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.527275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c02a67-99f6-4404-bee4-fde073ba2e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.529293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c02a67-99f6-4404-bee4-fde073ba2e93 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.539508 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1205/2383] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:48.543669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c02a67-99f6-4404-bee4-fde073ba2e93 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.553271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c02a67-99f6-4404-bee4-fde073ba2e93 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.558287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26088a0-e85c-46cb-a809-e4a67f235cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.558433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e702aa-8750-4096-81da-669a8701d6ae None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.558792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26088a0-e85c-46cb-a809-e4a67f235cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.559205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26088a0-e85c-46cb-a809-e4a67f235cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.559595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26088a0-e85c-46cb-a809-e4a67f235cf9 admin admin] PATH_INFO: `/v3/projects/c1de4096b1724f38b6bc8b4a1a97a309/users/befde0e61d074559a4449feb517690e9/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.561931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8494448-fc62-48fa-88db-f136ee6c2a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.562512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8494448-fc62-48fa-88db-f136ee6c2a30 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.562789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8494448-fc62-48fa-88db-f136ee6c2a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.563058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8494448-fc62-48fa-88db-f136ee6c2a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.572915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8494448-fc62-48fa-88db-f136ee6c2a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.574462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8494448-fc62-48fa-88db-f136ee6c2a30 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.575999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26088a0-e85c-46cb-a809-e4a67f235cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1de4096b1724f38b6bc8b4a1a97a309, user_id=befde0e61d074559a4449feb517690e9, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.578714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26088a0-e85c-46cb-a809-e4a67f235cf9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.585259 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1179/2384] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:48.593438 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1206/2385] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:48 2022] PUT /identity/v3/projects/c1de4096b1724f38b6bc8b4a1a97a309/users/befde0e61d074559a4449feb517690e9/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:48.598430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8494448-fc62-48fa-88db-f136ee6c2a30 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.610175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8494448-fc62-48fa-88db-f136ee6c2a30 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.686863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29167e67-bad8-4f43-bd62-c4d2045b8784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.686863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167e67-bad8-4f43-bd62-c4d2045b8784 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.686863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167e67-bad8-4f43-bd62-c4d2045b8784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.686863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167e67-bad8-4f43-bd62-c4d2045b8784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.687426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29167e67-bad8-4f43-bd62-c4d2045b8784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.687426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29167e67-bad8-4f43-bd62-c4d2045b8784 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.700047 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1180/2386] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:48.705015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26088a0-e85c-46cb-a809-e4a67f235cf9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.717135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26088a0-e85c-46cb-a809-e4a67f235cf9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:48.722922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:48.723587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.724003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.724289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] PATH_INFO: `/v3/projects/c1de4096b1724f38b6bc8b4a1a97a309/users/befde0e61d074559a4449feb517690e9/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.726516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1de4096b1724f38b6bc8b4a1a97a309, user_id=befde0e61d074559a4449feb517690e9, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:48.729347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:48.746328 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1207/2387] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:48 2022] PUT /identity/v3/projects/c1de4096b1724f38b6bc8b4a1a97a309/users/befde0e61d074559a4449feb517690e9/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:48.751630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba58ab2-ec57-46b4-8746-bb85325c559d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:48.752145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba58ab2-ec57-46b4-8746-bb85325c559d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:48.752250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba58ab2-ec57-46b4-8746-bb85325c559d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:48.801282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-eba58ab2-ec57-46b4-8746-bb85325c559d None None] MFA Rules not processed for user `befde0e61d074559a4449feb517690e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:48.803179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eba58ab2-ec57-46b4-8746-bb85325c559d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:48.905147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:48.920984 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1181/2388] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:48.938576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.941395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:48.990387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba58ab2-ec57-46b4-8746-bb85325c559d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:48.992781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eba58ab2-ec57-46b4-8746-bb85325c559d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:49.022056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d6a3e-8636-4dda-a0e1-40ddbe74024a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:49.027184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:49.027810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:49.028092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:49.028365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:49.030364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:49.085891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba58ab2-ec57-46b4-8746-bb85325c559d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:49.092112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:49.092887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:49.093326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:49.093692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] PATH_INFO: `/v3/projects/058c4b19eabb472ba8f41a63c65cbac0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:49.094970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=058c4b19eabb472ba8f41a63c65cbac0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:49.097164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:49.097501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:49.099181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:49.108477 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1208/2389] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:49.150486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058c4b19eabb472ba8f41a63c65cbac0', 'request_id': 'req-970bc81d-6653-4b3d-b985-0a9687bce4f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:49.161319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058c4b19eabb472ba8f41a63c65cbac0', 'request_id': 'req-970bc81d-6653-4b3d-b985-0a9687bce4f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:49.167759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058c4b19eabb472ba8f41a63c65cbac0', 'request_id': 'req-970bc81d-6653-4b3d-b985-0a9687bce4f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:49.180366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] The token cache is being invalidate because project 058c4b19eabb472ba8f41a63c65cbac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:49.180824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:49.181792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-970bc81d-6653-4b3d-b985-0a9687bce4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:49.183459 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1182/2390] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:48 2022] DELETE /identity/v3/projects/058c4b19eabb472ba8f41a63c65cbac0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:37:51.720185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:51.731667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc7361-4ae8-4cff-bed4-36099856a328 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:51.794552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:51.806648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:51.807356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:51.807691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:51.808390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] PATH_INFO: `/v3/users/095cf66d04004103801fb93676892003` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:51.809293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=095cf66d04004103801fb93676892003)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:51.811206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:51.868226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '095cf66d04004103801fb93676892003'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:51.875330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '095cf66d04004103801fb93676892003'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:51.876072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '095cf66d04004103801fb93676892003'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:51.876701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '095cf66d04004103801fb93676892003'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:51.885181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '095cf66d04004103801fb93676892003'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:51.903958 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1209/2391] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:51 2022] DELETE /identity/v3/users/095cf66d04004103801fb93676892003 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:51.908478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e96a6a-aa50-4bd7-af21-5dc9b6628381 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:51.909593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e96a6a-aa50-4bd7-af21-5dc9b6628381 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:51.909780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e96a6a-aa50-4bd7-af21-5dc9b6628381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:51.929657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-49e96a6a-aa50-4bd7-af21-5dc9b6628381 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:51.930903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-49e96a6a-aa50-4bd7-af21-5dc9b6628381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:52.000370 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1183/2392] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:52.011030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:52.012411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:52.078394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2758d-b817-4bf9-bfd0-0e40a1016dfd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:52.082320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efea71-b990-422a-a4a1-67df3aaf179e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:52.082955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efea71-b990-422a-a4a1-67df3aaf179e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:52.083229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efea71-b990-422a-a4a1-67df3aaf179e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:52.083500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efea71-b990-422a-a4a1-67df3aaf179e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:52.096109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efea71-b990-422a-a4a1-67df3aaf179e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:52.097600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efea71-b990-422a-a4a1-67df3aaf179e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:52.107844 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1210/2393] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:52.383411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e96a6a-aa50-4bd7-af21-5dc9b6628381 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:52.384858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-49e96a6a-aa50-4bd7-af21-5dc9b6628381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:52.412061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e96a6a-aa50-4bd7-af21-5dc9b6628381 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:52.417026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:52.417703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:52.417987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:52.418265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] PATH_INFO: `/v3/projects/7550f3c2ad3d4123a750d76fbe7ae9a3` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:52.419392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7550f3c2ad3d4123a750d76fbe7ae9a3)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:52.421347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:52.443142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:52.468221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7550f3c2ad3d4123a750d76fbe7ae9a3', 'request_id': 'req-2ff82505-283f-42b0-9a0e-eadfedd16d4b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.476224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7550f3c2ad3d4123a750d76fbe7ae9a3', 'request_id': 'req-2ff82505-283f-42b0-9a0e-eadfedd16d4b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.482307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7550f3c2ad3d4123a750d76fbe7ae9a3', 'request_id': 'req-2ff82505-283f-42b0-9a0e-eadfedd16d4b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.498143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] The token cache is being invalidate because project 7550f3c2ad3d4123a750d76fbe7ae9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:52.498517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.499188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.500688 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1184/2394] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:52 2022] DELETE /identity/v3/projects/7550f3c2ad3d4123a750d76fbe7ae9a3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:52.505059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efea71-b990-422a-a4a1-67df3aaf179e service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:52.515073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efea71-b990-422a-a4a1-67df3aaf179e service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:52.576029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:52.576666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:52.576968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:52.577238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] PATH_INFO: `/v3/users/befde0e61d074559a4449feb517690e9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:52.578382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] RBAC: Authorizing `identity:delete_user(user_id=befde0e61d074559a4449feb517690e9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:52.580293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:52.603862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:52.633939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'befde0e61d074559a4449feb517690e9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.641420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'befde0e61d074559a4449feb517690e9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.642251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'befde0e61d074559a4449feb517690e9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.643090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'befde0e61d074559a4449feb517690e9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.651919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-73ce9bb3-1430-4ab7-8bf4-ae8cc4c8c40c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'befde0e61d074559a4449feb517690e9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:52.669097 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1211/2395] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:37:52 2022] DELETE /identity/v3/users/befde0e61d074559a4449feb517690e9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:53.184049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:53.185644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:53.200545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff82505-283f-42b0-9a0e-eadfedd16d4b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:53.267324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:53.267442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.267669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.267958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] PATH_INFO: `/v3/projects/c1de4096b1724f38b6bc8b4a1a97a309` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.270037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1de4096b1724f38b6bc8b4a1a97a309)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:53.272347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:53.322806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1de4096b1724f38b6bc8b4a1a97a309', 'request_id': 'req-58e297b3-4c65-4e76-9a52-95966ee66f6d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:53.330076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1de4096b1724f38b6bc8b4a1a97a309', 'request_id': 'req-58e297b3-4c65-4e76-9a52-95966ee66f6d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:53.336679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1de4096b1724f38b6bc8b4a1a97a309', 'request_id': 'req-58e297b3-4c65-4e76-9a52-95966ee66f6d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:53.347126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] The token cache is being invalidate because project c1de4096b1724f38b6bc8b4a1a97a309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:37:53.347417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:53.348091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-58e297b3-4c65-4e76-9a52-95966ee66f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:37:53.349588 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1185/2396] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:37:53 2022] DELETE /identity/v3/projects/c1de4096b1724f38b6bc8b4a1a97a309 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:53.354421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb1c6-bedb-458f-b0dd-647eae011eab None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.354874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb1c6-bedb-458f-b0dd-647eae011eab None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.354981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cb1c6-bedb-458f-b0dd-647eae011eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.374643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5c1cb1c6-bedb-458f-b0dd-647eae011eab None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:53.375834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5c1cb1c6-bedb-458f-b0dd-647eae011eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:53.445098 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1212/2397] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:53.453861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672dbc2-2ffd-4560-b233-b0a521d7a820 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.454147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672dbc2-2ffd-4560-b233-b0a521d7a820 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.454425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672dbc2-2ffd-4560-b233-b0a521d7a820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.474637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2672dbc2-2ffd-4560-b233-b0a521d7a820 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:53.475854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2672dbc2-2ffd-4560-b233-b0a521d7a820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:53.483921 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1186/2398] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:53.489314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cb1c6-bedb-458f-b0dd-647eae011eab None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:53.499463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cb1c6-bedb-458f-b0dd-647eae011eab None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:53.504995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:53.505607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.505927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.506203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.506857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:37:53.507196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:53.508816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:53.521552 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1213/2399] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:37:53.525883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672dbc2-2ffd-4560-b233-b0a521d7a820 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:53.535271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672dbc2-2ffd-4560-b233-b0a521d7a820 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:53.539972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb368-062c-4b0b-a51d-4d36c2f78e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:53.540581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb368-062c-4b0b-a51d-4d36c2f78e80 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.540862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb368-062c-4b0b-a51d-4d36c2f78e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.541140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb368-062c-4b0b-a51d-4d36c2f78e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.542009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdb368-062c-4b0b-a51d-4d36c2f78e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:53.543914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdb368-062c-4b0b-a51d-4d36c2f78e80 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:53.573811 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1187/2400] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:37:53 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:53.577858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:53.589985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c51e0-e0b4-4273-879a-b498ce8a7abe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:53.648507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a4ed-c26f-4117-a8cc-ab98a809b43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:53.649118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a4ed-c26f-4117-a8cc-ab98a809b43a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.649399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a4ed-c26f-4117-a8cc-ab98a809b43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.649695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a4ed-c26f-4117-a8cc-ab98a809b43a admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.650335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0a4ed-c26f-4117-a8cc-ab98a809b43a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:53.652253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0a4ed-c26f-4117-a8cc-ab98a809b43a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:53.679098 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1214/2401] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:37:53 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:37:53.683255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb368-062c-4b0b-a51d-4d36c2f78e80 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:53.694432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb368-062c-4b0b-a51d-4d36c2f78e80 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:53.700020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ec7f-8099-4ee0-b38e-c0a88b16483f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:53.700020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ec7f-8099-4ee0-b38e-c0a88b16483f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.700280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ec7f-8099-4ee0-b38e-c0a88b16483f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.700401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ec7f-8099-4ee0-b38e-c0a88b16483f admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.701055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6ec7f-8099-4ee0-b38e-c0a88b16483f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:53.702681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6ec7f-8099-4ee0-b38e-c0a88b16483f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:53.705218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:53.718452 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1188/2402] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:37:53.719197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a4ed-c26f-4117-a8cc-ab98a809b43a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:53.730227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a4ed-c26f-4117-a8cc-ab98a809b43a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:53.734824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ce299-4850-4bfc-96d9-cbac299e6da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:53.735412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce299-4850-4bfc-96d9-cbac299e6da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.735692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce299-4850-4bfc-96d9-cbac299e6da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.735964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ce299-4850-4bfc-96d9-cbac299e6da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.736607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ce299-4850-4bfc-96d9-cbac299e6da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:53.738682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ce299-4850-4bfc-96d9-cbac299e6da9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:53.743882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:37:53.750399 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1215/2403] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:37:53.754805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ec7f-8099-4ee0-b38e-c0a88b16483f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:53.764712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ec7f-8099-4ee0-b38e-c0a88b16483f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:53.770312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:53.771124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.771605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.772071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] PATH_INFO: `/v3/projects/a530d9ebb5c749d2adaf3c428c5a4189/users/ef3be0cdb77b44d09443f3ecf8e63b79/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.792302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a530d9ebb5c749d2adaf3c428c5a4189, user_id=ef3be0cdb77b44d09443f3ecf8e63b79, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:53.795028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:53.807571 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1189/2404] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:37:53 2022] PUT /identity/v3/projects/a530d9ebb5c749d2adaf3c428c5a4189/users/ef3be0cdb77b44d09443f3ecf8e63b79/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:37:53.812628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b0025-f474-4570-88ac-1df2b3fd542d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.812914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b0025-f474-4570-88ac-1df2b3fd542d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:53.813853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b0025-f474-4570-88ac-1df2b3fd542d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:53.839307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1d6b0025-f474-4570-88ac-1df2b3fd542d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:53.840553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1d6b0025-f474-4570-88ac-1df2b3fd542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:53.910772 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1216/2405] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:53.920483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:53.922223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:53.993988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73208d-4eb8-4406-be0d-b3a1436a6622 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:53.998967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:53.999598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:53.999881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:54.000318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:54.010405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:54.011844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:54.023249 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1190/2406] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:37:54.978806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ba48c-2b15-4a3a-9b69-760321eeb702 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:54.979585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ba48c-2b15-4a3a-9b69-760321eeb702 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:54.979781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ba48c-2b15-4a3a-9b69-760321eeb702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:55.010536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-323ba48c-2b15-4a3a-9b69-760321eeb702 None None] MFA Rules not processed for user `ef3be0cdb77b44d09443f3ecf8e63b79`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:55.011917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-323ba48c-2b15-4a3a-9b69-760321eeb702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:55.086570 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1217/2407] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:55.098770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:55.100256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:55.163824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3b319-f590-42dc-9e5d-5359084e2f8f service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:55.167578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d9701-0659-464d-acc8-f5341c7810af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:55.168108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9701-0659-464d-acc8-f5341c7810af service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:55.168394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9701-0659-464d-acc8-f5341c7810af service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:55.168669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d9701-0659-464d-acc8-f5341c7810af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:55.178037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d9701-0659-464d-acc8-f5341c7810af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:55.179582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d9701-0659-464d-acc8-f5341c7810af service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:55.189940 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1191/2408] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:37:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:37:56.626364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810fd2-617b-4171-976a-4750af92ee24 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:56.626796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810fd2-617b-4171-976a-4750af92ee24 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:56.626905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810fd2-617b-4171-976a-4750af92ee24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:56.773245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-78810fd2-617b-4171-976a-4750af92ee24 None None] MFA Rules not processed for user `ef3be0cdb77b44d09443f3ecf8e63b79`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:56.774526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-78810fd2-617b-4171-976a-4750af92ee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:56.785738 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1218/2409] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:37:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:59.491185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d39883-7f96-421a-b0c4-e3faad628b61 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:59.491768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d39883-7f96-421a-b0c4-e3faad628b61 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:59.491768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d39883-7f96-421a-b0c4-e3faad628b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:59.511693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-35d39883-7f96-421a-b0c4-e3faad628b61 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:37:59.512884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-35d39883-7f96-421a-b0c4-e3faad628b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:59.570889 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1192/2410] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:37:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:37:59.614365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78810fd2-617b-4171-976a-4750af92ee24 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:37:59.615950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-78810fd2-617b-4171-976a-4750af92ee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:37:59.686195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78810fd2-617b-4171-976a-4750af92ee24 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:37:59.692732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b3df3-9a5b-44a5-aec6-bb35bb5bdf9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:37:59.693385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b3df3-9a5b-44a5-aec6-bb35bb5bdf9a service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:37:59.693677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b3df3-9a5b-44a5-aec6-bb35bb5bdf9a service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:37:59.694052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b3df3-9a5b-44a5-aec6-bb35bb5bdf9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:37:59.702283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b3df3-9a5b-44a5-aec6-bb35bb5bdf9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:37:59.703862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b3df3-9a5b-44a5-aec6-bb35bb5bdf9a service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:37:59.712493 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1219/2411] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:37:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 30 09:38:00.232525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f02074-b0fb-4617-94a8-ce7cbedcee23 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:00.232780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f02074-b0fb-4617-94a8-ce7cbedcee23 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:00.233055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f02074-b0fb-4617-94a8-ce7cbedcee23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:00.253960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-87f02074-b0fb-4617-94a8-ce7cbedcee23 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:00.255284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-87f02074-b0fb-4617-94a8-ce7cbedcee23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:00.263266 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1193/2412] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 30 09:38:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:00.278671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b3df3-9a5b-44a5-aec6-bb35bb5bdf9a service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:00.291330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b3df3-9a5b-44a5-aec6-bb35bb5bdf9a service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:00.297971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea236013-92d4-4fa1-86c6-1ea888a253da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:00.298567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236013-92d4-4fa1-86c6-1ea888a253da service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:00.298905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236013-92d4-4fa1-86c6-1ea888a253da service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:00.299215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236013-92d4-4fa1-86c6-1ea888a253da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:00.310648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea236013-92d4-4fa1-86c6-1ea888a253da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:00.312284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea236013-92d4-4fa1-86c6-1ea888a253da service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:00.318130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:00.333714 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1220/2413] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:38:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:38:00.661297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f7f8-3df0-425b-85f0-e92230ad6769 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:00.661821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f7f8-3df0-425b-85f0-e92230ad6769 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:00.662045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f7f8-3df0-425b-85f0-e92230ad6769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:00.685775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ef07f7f8-3df0-425b-85f0-e92230ad6769 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:00.687211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ef07f7f8-3df0-425b-85f0-e92230ad6769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:00.695730 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1194/2414] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:38:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:00.708375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea236013-92d4-4fa1-86c6-1ea888a253da service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:00.719332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea236013-92d4-4fa1-86c6-1ea888a253da service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:00.730469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba83ad9-dae4-4441-b0ab-527eb7bd1dc2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:00.732510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba83ad9-dae4-4441-b0ab-527eb7bd1dc2 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:00.732980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba83ad9-dae4-4441-b0ab-527eb7bd1dc2 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:00.733258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba83ad9-dae4-4441-b0ab-527eb7bd1dc2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:00.743599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba83ad9-dae4-4441-b0ab-527eb7bd1dc2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:00.745200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba83ad9-dae4-4441-b0ab-527eb7bd1dc2 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:00.754699 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1221/2415] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:38:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:38:01.213439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba2ed-99d0-444f-8a1a-0096bcfb6a66 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:01.213832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba2ed-99d0-444f-8a1a-0096bcfb6a66 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:01.214186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ba2ed-99d0-444f-8a1a-0096bcfb6a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:01.235700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-af5ba2ed-99d0-444f-8a1a-0096bcfb6a66 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:01.237718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-af5ba2ed-99d0-444f-8a1a-0096bcfb6a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:01.245150 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1195/2416] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 30 09:38:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:38:01.258710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba83ad9-dae4-4441-b0ab-527eb7bd1dc2 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:01.272252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba83ad9-dae4-4441-b0ab-527eb7bd1dc2 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:01.277988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793b43f-4dbf-4459-b4cb-04d4cdaf8d35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:01.278770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b43f-4dbf-4459-b4cb-04d4cdaf8d35 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:01.279060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b43f-4dbf-4459-b4cb-04d4cdaf8d35 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:01.279334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b43f-4dbf-4459-b4cb-04d4cdaf8d35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:01.288253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793b43f-4dbf-4459-b4cb-04d4cdaf8d35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:01.290052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793b43f-4dbf-4459-b4cb-04d4cdaf8d35 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:01.298823 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1222/2417] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:38:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:38:03.672460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ba2ed-99d0-444f-8a1a-0096bcfb6a66 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:03.673949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-af5ba2ed-99d0-444f-8a1a-0096bcfb6a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:03.683684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:03.688248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ba2ed-99d0-444f-8a1a-0096bcfb6a66 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:03.694445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:03.695077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:03.695376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:03.695647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] PATH_INFO: `/v3/users/9f38a5ac076c4e9d841095bdca705c14` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:03.696779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f38a5ac076c4e9d841095bdca705c14)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:03.698786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:03.756289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f38a5ac076c4e9d841095bdca705c14'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:03.763935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f38a5ac076c4e9d841095bdca705c14'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:03.766015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f38a5ac076c4e9d841095bdca705c14'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:03.766834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f38a5ac076c4e9d841095bdca705c14'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:03.779180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f38a5ac076c4e9d841095bdca705c14'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:03.811236 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1196/2418] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:38:03 2022] DELETE /identity/v3/users/9f38a5ac076c4e9d841095bdca705c14 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:03.816044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b770-ddca-41eb-a5c4-d10c8a27a88f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:03.816701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b770-ddca-41eb-a5c4-d10c8a27a88f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:03.816701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7b770-ddca-41eb-a5c4-d10c8a27a88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:03.835701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f5b7b770-ddca-41eb-a5c4-d10c8a27a88f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:03.836905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f5b7b770-ddca-41eb-a5c4-d10c8a27a88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:03.915153 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1223/2419] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:03.927988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:03.929422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:04.003609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c2b58-5431-4c3c-916a-f2c11dd73d32 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:04.007732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d0cd9-fdac-4fb5-bf1b-bd4718b1dc22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:04.008378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d0cd9-fdac-4fb5-bf1b-bd4718b1dc22 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:04.008613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d0cd9-fdac-4fb5-bf1b-bd4718b1dc22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:04.008887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d0cd9-fdac-4fb5-bf1b-bd4718b1dc22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:04.019436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d0cd9-fdac-4fb5-bf1b-bd4718b1dc22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:04.021157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d0cd9-fdac-4fb5-bf1b-bd4718b1dc22 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:04.035249 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1197/2420] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:38:04.251211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7b770-ddca-41eb-a5c4-d10c8a27a88f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:04.252510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f5b7b770-ddca-41eb-a5c4-d10c8a27a88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:04.266430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7b770-ddca-41eb-a5c4-d10c8a27a88f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:04.271976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:04.273082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:04.273728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:04.274249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] PATH_INFO: `/v3/projects/0124ee0c926240e8b69b3b70ef93080b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:04.275599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0124ee0c926240e8b69b3b70ef93080b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:04.278939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:04.343271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0124ee0c926240e8b69b3b70ef93080b', 'request_id': 'req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:04.351161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0124ee0c926240e8b69b3b70ef93080b', 'request_id': 'req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:04.357800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0124ee0c926240e8b69b3b70ef93080b', 'request_id': 'req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:04.369733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] The token cache is being invalidate because project 0124ee0c926240e8b69b3b70ef93080b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:38:04.370035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:04.370687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:04.372160 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1224/2421] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:38:04 2022] DELETE /identity/v3/projects/0124ee0c926240e8b69b3b70ef93080b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:14.026871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:14.366654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:24.078214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208135cc-f8d0-4a3e-a94f-55765030670b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:24.078718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208135cc-f8d0-4a3e-a94f-55765030670b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:24.078718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208135cc-f8d0-4a3e-a94f-55765030670b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:24.100693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-208135cc-f8d0-4a3e-a94f-55765030670b None None] MFA Rules not processed for user `ef3be0cdb77b44d09443f3ecf8e63b79`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:24.101967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-208135cc-f8d0-4a3e-a94f-55765030670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:24.157432 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1198/2422] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:27.443973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:27.445462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:27.491815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208135cc-f8d0-4a3e-a94f-55765030670b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:27.494249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-208135cc-f8d0-4a3e-a94f-55765030670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:27.538444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89838e-2ecd-45bd-95bc-cf8fcdad93b2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:27.544861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:27.545657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:27.546053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:27.546529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] PATH_INFO: `/v3/users/7b71b56279a94ad5bde3fb2a66bde7a8` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:27.549242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b71b56279a94ad5bde3fb2a66bde7a8)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:27.551384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:27.594648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208135cc-f8d0-4a3e-a94f-55765030670b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:27.600643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:27.601491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:27.602150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:27.602635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] PATH_INFO: `/v3/users/f1e20302069242f5b7dcd2647c977057` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:27.604187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e20302069242f5b7dcd2647c977057)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:27.606639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:27.623250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b71b56279a94ad5bde3fb2a66bde7a8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.633978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b71b56279a94ad5bde3fb2a66bde7a8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.634876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b71b56279a94ad5bde3fb2a66bde7a8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.638487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b71b56279a94ad5bde3fb2a66bde7a8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.650753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-826eb7e3-61e2-4292-a890-a9ea15725ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b71b56279a94ad5bde3fb2a66bde7a8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.684730 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1225/2423] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:38:27 2022] DELETE /identity/v3/users/7b71b56279a94ad5bde3fb2a66bde7a8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:27.690089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ed1a7-1e17-4255-afc1-f3e2de3431ab None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:27.690089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ed1a7-1e17-4255-afc1-f3e2de3431ab None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:27.690334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ed1a7-1e17-4255-afc1-f3e2de3431ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:27.698047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e20302069242f5b7dcd2647c977057'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.710897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e80ed1a7-1e17-4255-afc1-f3e2de3431ab None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:27.712135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e80ed1a7-1e17-4255-afc1-f3e2de3431ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:27.722919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e20302069242f5b7dcd2647c977057'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.725117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e20302069242f5b7dcd2647c977057'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e20302069242f5b7dcd2647c977057'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.739455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76cc2d0e-f046-4949-a114-f9ee81a30c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e20302069242f5b7dcd2647c977057'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:27.768250 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1199/2424] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:38:27 2022] DELETE /identity/v3/users/f1e20302069242f5b7dcd2647c977057 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:27.769520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247f648-7fd0-4b0b-b5d3-367fba1e6577 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:27.770093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247f648-7fd0-4b0b-b5d3-367fba1e6577 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:27.770363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247f648-7fd0-4b0b-b5d3-367fba1e6577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:27.784514 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1226/2425] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:27.795439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6247f648-7fd0-4b0b-b5d3-367fba1e6577 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:27.796901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6247f648-7fd0-4b0b-b5d3-367fba1e6577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:27.800825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ed1a7-1e17-4255-afc1-f3e2de3431ab None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:27.805365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e80ed1a7-1e17-4255-afc1-f3e2de3431ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:27.820168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:27.846455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:27.918611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ed1a7-1e17-4255-afc1-f3e2de3431ab None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:27.923885 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1200/2426] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:27.939705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247f648-7fd0-4b0b-b5d3-367fba1e6577 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:28.012658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:28.013371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.013784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.014157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.027178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:28.028626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:28.039336 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1227/2427] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:38:28.041941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247f648-7fd0-4b0b-b5d3-367fba1e6577 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:28.046155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:28.046913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.047265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.047607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.059230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:28.060971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:28.071810 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1201/2428] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:38:28.696566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:28.698213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:28.716389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1816c9-39cc-4868-ae03-0eac07b3fab8 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:28.716842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:28.718385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:28.724482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:28.725819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.726097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.726366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] PATH_INFO: `/v3/projects/51c74da8d75b4421a3ac2440584f1551` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.727587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c74da8d75b4421a3ac2440584f1551)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:28.729480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:28.734091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f07abb-8865-4f9c-bdf5-ad53f05f7edd service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:28.740412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:28.741120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.741397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.741861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] PATH_INFO: `/v3/projects/93c1c087dc60448e82e206f3082e58ff` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.743145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c1c087dc60448e82e206f3082e58ff)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:28.745405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:28.783983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c74da8d75b4421a3ac2440584f1551', 'request_id': 'req-2576afe3-8688-4bb6-a46a-60d784beff3f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.791776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c74da8d75b4421a3ac2440584f1551', 'request_id': 'req-2576afe3-8688-4bb6-a46a-60d784beff3f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.796933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c1c087dc60448e82e206f3082e58ff', 'request_id': 'req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.798648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c74da8d75b4421a3ac2440584f1551', 'request_id': 'req-2576afe3-8688-4bb6-a46a-60d784beff3f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.806431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c1c087dc60448e82e206f3082e58ff', 'request_id': 'req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.809621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] The token cache is being invalidate because project 51c74da8d75b4421a3ac2440584f1551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:38:28.809932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.810612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2576afe3-8688-4bb6-a46a-60d784beff3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.812119 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1228/2429] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:38:28 2022] DELETE /identity/v3/projects/51c74da8d75b4421a3ac2440584f1551 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:28.815267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c1c087dc60448e82e206f3082e58ff', 'request_id': 'req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.817819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c0ef6-6ee8-40aa-aed0-212e9642ce55 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.818093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c0ef6-6ee8-40aa-aed0-212e9642ce55 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.818367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c0ef6-6ee8-40aa-aed0-212e9642ce55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.827633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] The token cache is being invalidate because project 93c1c087dc60448e82e206f3082e58ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:38:28.827930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.828589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fcbd66a8-2721-4ec3-ab24-2e28bd26893c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:28.830187 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1202/2430] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:38:28 2022] DELETE /identity/v3/projects/93c1c087dc60448e82e206f3082e58ff => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:28.834884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff21b-ff3d-482c-a5cb-4f1f572576d8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.835225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff21b-ff3d-482c-a5cb-4f1f572576d8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.835580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff21b-ff3d-482c-a5cb-4f1f572576d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.838458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ea2c0ef6-6ee8-40aa-aed0-212e9642ce55 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:28.839655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ea2c0ef6-6ee8-40aa-aed0-212e9642ce55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:28.855129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-850ff21b-ff3d-482c-a5cb-4f1f572576d8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:28.856327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-850ff21b-ff3d-482c-a5cb-4f1f572576d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:28.900368 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1229/2431] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:38:28.903654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:28.908641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424082-0589-44ad-9aa3-b117bc394434 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.909015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424082-0589-44ad-9aa3-b117bc394434 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.909318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c424082-0589-44ad-9aa3-b117bc394434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.917202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:28.921768 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1203/2432] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:38:28.930378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424640d-6dfb-4f40-81ac-36a8ffdd4e85 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.930667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424640d-6dfb-4f40-81ac-36a8ffdd4e85 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.930868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1c424082-0589-44ad-9aa3-b117bc394434 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:28.930979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424640d-6dfb-4f40-81ac-36a8ffdd4e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.932691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1c424082-0589-44ad-9aa3-b117bc394434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:28.940737 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1230/2433] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:38:28.944846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c424082-0589-44ad-9aa3-b117bc394434 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:28.950345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6424640d-6dfb-4f40-81ac-36a8ffdd4e85 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:28.951545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6424640d-6dfb-4f40-81ac-36a8ffdd4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:28.953950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c424082-0589-44ad-9aa3-b117bc394434 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:28.958837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:28.959421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.959706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.959835 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1204/2434] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:38:28.959982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.960697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:38:28.961004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:28.963111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:28.964019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424640d-6dfb-4f40-81ac-36a8ffdd4e85 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:28.974138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424640d-6dfb-4f40-81ac-36a8ffdd4e85 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:28.977853 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1231/2435] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:38:28.979852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:28.980221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:28.981013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.981286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.981551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.982230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:38:28.982534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:28.984315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:28.991484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5369f1-d3ce-4c49-860a-ba111b8494fe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:28.996540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98daa-ce9e-4183-8467-ab7c2f30772b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:28.997171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98daa-ce9e-4183-8467-ab7c2f30772b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:28.997452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98daa-ce9e-4183-8467-ab7c2f30772b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:28.997760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98daa-ce9e-4183-8467-ab7c2f30772b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:28.998411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e98daa-ce9e-4183-8467-ab7c2f30772b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.003441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e98daa-ce9e-4183-8467-ab7c2f30772b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.003675 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1205/2436] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:38:29.004717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.017589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c509f-df06-4f94-83d7-4fb820b8c5bd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.024415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cc403-f0c2-4619-9d3e-c8062571348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.025110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cc403-f0c2-4619-9d3e-c8062571348e admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.025394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cc403-f0c2-4619-9d3e-c8062571348e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.025975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cc403-f0c2-4619-9d3e-c8062571348e admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.026398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cc403-f0c2-4619-9d3e-c8062571348e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.028406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cc403-f0c2-4619-9d3e-c8062571348e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.037498 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1232/2437] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:38:28 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:38:29.038685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98daa-ce9e-4183-8467-ab7c2f30772b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.065823 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1206/2438] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:38:29 2022] POST /identity/v3/projects => generated 399 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:38:29.067003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cc403-f0c2-4619-9d3e-c8062571348e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.119639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98daa-ce9e-4183-8467-ab7c2f30772b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.148482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cc403-f0c2-4619-9d3e-c8062571348e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.165523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71523c50-29c0-4868-a6a3-732f004c8070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.166192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71523c50-29c0-4868-a6a3-732f004c8070 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.166479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71523c50-29c0-4868-a6a3-732f004c8070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.166757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71523c50-29c0-4868-a6a3-732f004c8070 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.167406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71523c50-29c0-4868-a6a3-732f004c8070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.169394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71523c50-29c0-4868-a6a3-732f004c8070 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.193984 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1207/2439] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:38:29 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:38:29.197946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71523c50-29c0-4868-a6a3-732f004c8070 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.209919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71523c50-29c0-4868-a6a3-732f004c8070 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.213777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ad8c0-0fe4-40a3-9c1f-a5e188f48e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.214462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ad8c0-0fe4-40a3-9c1f-a5e188f48e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.214818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ad8c0-0fe4-40a3-9c1f-a5e188f48e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.215170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ad8c0-0fe4-40a3-9c1f-a5e188f48e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.215740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25854f0-969a-425d-a207-f2537807d00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.215944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ad8c0-0fe4-40a3-9c1f-a5e188f48e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.216375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25854f0-969a-425d-a207-f2537807d00b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.216658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25854f0-969a-425d-a207-f2537807d00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.216933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25854f0-969a-425d-a207-f2537807d00b admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.217580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25854f0-969a-425d-a207-f2537807d00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.217971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ad8c0-0fe4-40a3-9c1f-a5e188f48e8c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.219223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25854f0-969a-425d-a207-f2537807d00b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.232269 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1208/2440] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:38:29.238536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25854f0-969a-425d-a207-f2537807d00b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.245212 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1233/2441] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:38:29 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:38:29.249368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ad8c0-0fe4-40a3-9c1f-a5e188f48e8c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.250910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25854f0-969a-425d-a207-f2537807d00b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.255853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912822e4-85ed-4a7d-97e2-1a9bd7d30977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.256354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912822e4-85ed-4a7d-97e2-1a9bd7d30977 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.256627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912822e4-85ed-4a7d-97e2-1a9bd7d30977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.256896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912822e4-85ed-4a7d-97e2-1a9bd7d30977 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.257538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912822e4-85ed-4a7d-97e2-1a9bd7d30977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.259220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912822e4-85ed-4a7d-97e2-1a9bd7d30977 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.262709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ad8c0-0fe4-40a3-9c1f-a5e188f48e8c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.270416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f77a43-da85-466c-9ce1-2ee8ade955ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.271030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f77a43-da85-466c-9ce1-2ee8ade955ca admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.271313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f77a43-da85-466c-9ce1-2ee8ade955ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.271589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f77a43-da85-466c-9ce1-2ee8ade955ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.272240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f77a43-da85-466c-9ce1-2ee8ade955ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.272818 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1209/2442] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:38:29.274136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f77a43-da85-466c-9ce1-2ee8ade955ca admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.276741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912822e4-85ed-4a7d-97e2-1a9bd7d30977 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.286168 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1234/2443] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:38:29.289562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912822e4-85ed-4a7d-97e2-1a9bd7d30977 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.290001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f77a43-da85-466c-9ce1-2ee8ade955ca admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.294314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aedeca-43e2-4274-bab7-15e16f86d0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.294969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aedeca-43e2-4274-bab7-15e16f86d0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.295246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aedeca-43e2-4274-bab7-15e16f86d0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.295519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aedeca-43e2-4274-bab7-15e16f86d0e4 admin admin] PATH_INFO: `/v3/projects/5543e63003fc434db21e24b2d44c1d5a/users/d86ab2dca5f14a389fe69ce020bb08c2/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.303349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f77a43-da85-466c-9ce1-2ee8ade955ca admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.308469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46c4e6-8509-47b4-ae7f-e551dbb42c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.309278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c4e6-8509-47b4-ae7f-e551dbb42c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.309672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c4e6-8509-47b4-ae7f-e551dbb42c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.310058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c4e6-8509-47b4-ae7f-e551dbb42c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.310794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46c4e6-8509-47b4-ae7f-e551dbb42c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.312474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46c4e6-8509-47b4-ae7f-e551dbb42c3a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.314818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aedeca-43e2-4274-bab7-15e16f86d0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5543e63003fc434db21e24b2d44c1d5a, user_id=d86ab2dca5f14a389fe69ce020bb08c2, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.317458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aedeca-43e2-4274-bab7-15e16f86d0e4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.324239 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1235/2444] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:38:29.328960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46c4e6-8509-47b4-ae7f-e551dbb42c3a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.333104 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1210/2445] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:38:29 2022] PUT /identity/v3/projects/5543e63003fc434db21e24b2d44c1d5a/users/d86ab2dca5f14a389fe69ce020bb08c2/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:38:29.340521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46c4e6-8509-47b4-ae7f-e551dbb42c3a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.378189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:29.405654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf5422-302b-46c7-9748-519874caba97 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.405934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf5422-302b-46c7-9748-519874caba97 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.406210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf5422-302b-46c7-9748-519874caba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.413346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47772a0-a8e1-4fbb-b7f7-34dfc9510bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.414051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47772a0-a8e1-4fbb-b7f7-34dfc9510bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.414332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47772a0-a8e1-4fbb-b7f7-34dfc9510bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.414604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47772a0-a8e1-4fbb-b7f7-34dfc9510bb1 admin admin] PATH_INFO: `/v3/projects/2c1cb4c57c924a24ab3368be3a3384cb/users/468fa04c3a11433aa8f6127b72b36c0c/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.436620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47772a0-a8e1-4fbb-b7f7-34dfc9510bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1cb4c57c924a24ab3368be3a3384cb, user_id=468fa04c3a11433aa8f6127b72b36c0c, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.437926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4ccf5422-302b-46c7-9748-519874caba97 None None] MFA Rules not processed for user `d86ab2dca5f14a389fe69ce020bb08c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:29.439563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4ccf5422-302b-46c7-9748-519874caba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:29.439885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47772a0-a8e1-4fbb-b7f7-34dfc9510bb1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.454082 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1236/2446] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:38:29 2022] PUT /identity/v3/projects/2c1cb4c57c924a24ab3368be3a3384cb/users/468fa04c3a11433aa8f6127b72b36c0c/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:29.459937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d90375-43b2-4526-b2dc-3ca0469ce3aa None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.460615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d90375-43b2-4526-b2dc-3ca0469ce3aa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.460886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d90375-43b2-4526-b2dc-3ca0469ce3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.480926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-75d90375-43b2-4526-b2dc-3ca0469ce3aa None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:29.482162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-75d90375-43b2-4526-b2dc-3ca0469ce3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:29.487631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:29.514876 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1211/2447] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:29 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:38:29.526519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf5422-302b-46c7-9748-519874caba97 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.528077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4ccf5422-302b-46c7-9748-519874caba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:29.556366 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1237/2448] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:29.566389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d90375-43b2-4526-b2dc-3ca0469ce3aa None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:29.568102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-75d90375-43b2-4526-b2dc-3ca0469ce3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:29.605923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf5422-302b-46c7-9748-519874caba97 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.652106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d90375-43b2-4526-b2dc-3ca0469ce3aa None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:29.656116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66f981-6cf7-4b65-8b2a-745f6ecb305b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.656811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f981-6cf7-4b65-8b2a-745f6ecb305b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.657153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f981-6cf7-4b65-8b2a-745f6ecb305b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.658025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f981-6cf7-4b65-8b2a-745f6ecb305b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.660778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8421da3-dcab-4ece-8a3b-fe52b944c410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:29.661381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8421da3-dcab-4ece-8a3b-fe52b944c410 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:29.661681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8421da3-dcab-4ece-8a3b-fe52b944c410 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:29.661950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8421da3-dcab-4ece-8a3b-fe52b944c410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:29.667743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66f981-6cf7-4b65-8b2a-745f6ecb305b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.669173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66f981-6cf7-4b65-8b2a-745f6ecb305b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.679555 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1238/2449] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:38:29.725093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8421da3-dcab-4ece-8a3b-fe52b944c410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:29.726572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8421da3-dcab-4ece-8a3b-fe52b944c410 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:29.737854 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1212/2450] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:38:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:38:34.911454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0da109-f8de-4631-a2c9-a60ae664981e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:34.912259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0da109-f8de-4631-a2c9-a60ae664981e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:34.912259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0da109-f8de-4631-a2c9-a60ae664981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:34.954096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ca0da109-f8de-4631-a2c9-a60ae664981e None None] MFA Rules not processed for user `468fa04c3a11433aa8f6127b72b36c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:34.955261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ca0da109-f8de-4631-a2c9-a60ae664981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:35.016878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8421da3-dcab-4ece-8a3b-fe52b944c410 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:35.026884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:35.030926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8421da3-dcab-4ece-8a3b-fe52b944c410 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:35.043143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:35.043894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.044160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.044432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] PATH_INFO: `/v3/users/d86ab2dca5f14a389fe69ce020bb08c2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.045620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86ab2dca5f14a389fe69ce020bb08c2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:35.047845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:35.056512 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1239/2451] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:38:35.058681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0da109-f8de-4631-a2c9-a60ae664981e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:35.071749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0da109-f8de-4631-a2c9-a60ae664981e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:35.076481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf4392-a5aa-4bdf-ab35-0760d1a2bfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:35.077100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4392-a5aa-4bdf-ab35-0760d1a2bfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.077375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4392-a5aa-4bdf-ab35-0760d1a2bfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.077671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf4392-a5aa-4bdf-ab35-0760d1a2bfb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.078641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf4392-a5aa-4bdf-ab35-0760d1a2bfb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:35.080538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf4392-a5aa-4bdf-ab35-0760d1a2bfb1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:35.130201 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1240/2452] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:38:35 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:38:35.130414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf4392-a5aa-4bdf-ab35-0760d1a2bfb1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:35.153964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86ab2dca5f14a389fe69ce020bb08c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:35.155799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:35.169704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86ab2dca5f14a389fe69ce020bb08c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:35.170474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86ab2dca5f14a389fe69ce020bb08c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:35.171093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86ab2dca5f14a389fe69ce020bb08c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:35.182315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-aab7bef4-2fd2-4d43-aa5f-ccd203c31009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86ab2dca5f14a389fe69ce020bb08c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:35.209715 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1213/2453] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:38:35 2022] DELETE /identity/v3/users/d86ab2dca5f14a389fe69ce020bb08c2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:35.213543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0190e-9844-4ceb-a48c-58e8a169d458 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.213994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0190e-9844-4ceb-a48c-58e8a169d458 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.214336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0190e-9844-4ceb-a48c-58e8a169d458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.231359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf4392-a5aa-4bdf-ab35-0760d1a2bfb1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:35.242132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c1d0190e-9844-4ceb-a48c-58e8a169d458 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:35.243528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c1d0190e-9844-4ceb-a48c-58e8a169d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:35.328611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:35.329279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.329854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.330335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.330975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:35.332912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:35.364786 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1241/2454] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:38:35 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:38:35.365109 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1214/2455] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:35.369862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:35.374606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:35.378885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0190e-9844-4ceb-a48c-58e8a169d458 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:35.380682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c1d0190e-9844-4ceb-a48c-58e8a169d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:35.395374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1bdf9-c227-469d-b72a-27df74907229 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:35.415620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410e270-ec5e-4741-9bef-ce3920511d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:35.416341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410e270-ec5e-4741-9bef-ce3920511d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.416719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410e270-ec5e-4741-9bef-ce3920511d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.417064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410e270-ec5e-4741-9bef-ce3920511d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.417867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410e270-ec5e-4741-9bef-ce3920511d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:35.419651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410e270-ec5e-4741-9bef-ce3920511d50 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:35.433422 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1242/2456] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:38:35.439081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410e270-ec5e-4741-9bef-ce3920511d50 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:35.454951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410e270-ec5e-4741-9bef-ce3920511d50 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:35.460962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb96ae-f46a-4b77-9de1-e368dc23aee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:35.461890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb96ae-f46a-4b77-9de1-e368dc23aee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.462589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb96ae-f46a-4b77-9de1-e368dc23aee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.462990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb96ae-f46a-4b77-9de1-e368dc23aee0 admin admin] PATH_INFO: `/v3/projects/39a6ad090122467da59af411d4ea3551/users/a7c72a2db99b4fe2a9c726c10bbe3af9/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.464548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0190e-9844-4ceb-a48c-58e8a169d458 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:35.469281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7506a-668d-4856-ad62-3cacf6c3c659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:35.470859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7506a-668d-4856-ad62-3cacf6c3c659 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.471395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7506a-668d-4856-ad62-3cacf6c3c659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.471827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7506a-668d-4856-ad62-3cacf6c3c659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.482656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7506a-668d-4856-ad62-3cacf6c3c659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:35.484375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7506a-668d-4856-ad62-3cacf6c3c659 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:35.487319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb96ae-f46a-4b77-9de1-e368dc23aee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a6ad090122467da59af411d4ea3551, user_id=a7c72a2db99b4fe2a9c726c10bbe3af9, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:35.490073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb96ae-f46a-4b77-9de1-e368dc23aee0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:35.496171 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1215/2457] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:38:35.509244 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1243/2458] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:38:35 2022] PUT /identity/v3/projects/39a6ad090122467da59af411d4ea3551/users/a7c72a2db99b4fe2a9c726c10bbe3af9/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:35.513238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7506a-668d-4856-ad62-3cacf6c3c659 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:35.524103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7506a-668d-4856-ad62-3cacf6c3c659 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:35.603371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:35.603891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.604173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.604500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.605527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:35.606882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:35.617868 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1216/2459] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:38:35.622331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb96ae-f46a-4b77-9de1-e368dc23aee0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:35.633673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb96ae-f46a-4b77-9de1-e368dc23aee0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:35.638183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab7a29-d9cf-4373-8046-28cf4e2cd07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:35.638851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7a29-d9cf-4373-8046-28cf4e2cd07e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:35.639131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7a29-d9cf-4373-8046-28cf4e2cd07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:35.639400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7a29-d9cf-4373-8046-28cf4e2cd07e admin admin] PATH_INFO: `/v3/projects/39a6ad090122467da59af411d4ea3551/users/a7c72a2db99b4fe2a9c726c10bbe3af9/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:35.641499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab7a29-d9cf-4373-8046-28cf4e2cd07e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a6ad090122467da59af411d4ea3551, user_id=a7c72a2db99b4fe2a9c726c10bbe3af9, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:35.645937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab7a29-d9cf-4373-8046-28cf4e2cd07e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:35.686806 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1244/2460] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:38:35 2022] PUT /identity/v3/projects/39a6ad090122467da59af411d4ea3551/users/a7c72a2db99b4fe2a9c726c10bbe3af9/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:36.118215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:36.120771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:36.140628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:36.145488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351225d-b7b6-4238-9c2c-73a16f33bf57 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:36.263615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:36.264426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:36.264944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:36.265318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] PATH_INFO: `/v3/projects/5543e63003fc434db21e24b2d44c1d5a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:36.267296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5543e63003fc434db21e24b2d44c1d5a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:36.269208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:36.330969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5543e63003fc434db21e24b2d44c1d5a', 'request_id': 'req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:36.346738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5543e63003fc434db21e24b2d44c1d5a', 'request_id': 'req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:36.357330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5543e63003fc434db21e24b2d44c1d5a', 'request_id': 'req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:36.374218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] The token cache is being invalidate because project 5543e63003fc434db21e24b2d44c1d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:38:36.375050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:36.376566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:38:36.421662 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1217/2461] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:38:36 2022] DELETE /identity/v3/projects/5543e63003fc434db21e24b2d44c1d5a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:38:41.228294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d8042-8475-4280-ab09-63476d1be898 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:41.229263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d8042-8475-4280-ab09-63476d1be898 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:41.231984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d8042-8475-4280-ab09-63476d1be898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:41.264291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-962d8042-8475-4280-ab09-63476d1be898 None None] MFA Rules not processed for user `a7c72a2db99b4fe2a9c726c10bbe3af9`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:41.265529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-962d8042-8475-4280-ab09-63476d1be898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:41.274937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:41.351360 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1245/2462] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:41.365041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:41.366629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:41.459252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb35c6-ed99-406a-8f18-5dacd47f0e0d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:41.467264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38b8d0-8bba-4791-814b-20ffed9e51ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:41.467921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38b8d0-8bba-4791-814b-20ffed9e51ed service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:41.468201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38b8d0-8bba-4791-814b-20ffed9e51ed service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:41.468486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38b8d0-8bba-4791-814b-20ffed9e51ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:41.470710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9c38b8d0-8bba-4791-814b-20ffed9e51ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:41.506268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:38:41.544027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38b8d0-8bba-4791-814b-20ffed9e51ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:41.546750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38b8d0-8bba-4791-814b-20ffed9e51ed service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:41.559667 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1218/2463] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:38:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:38:42.747053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210bf5-264e-42b2-ac6e-90f6a219e4fd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:42.747610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210bf5-264e-42b2-ac6e-90f6a219e4fd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:42.747610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87210bf5-264e-42b2-ac6e-90f6a219e4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:42.783510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-87210bf5-264e-42b2-ac6e-90f6a219e4fd None None] MFA Rules not processed for user `468fa04c3a11433aa8f6127b72b36c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:42.785223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-87210bf5-264e-42b2-ac6e-90f6a219e4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:42.799477 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1246/2464] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:48.976589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0117f5-5463-4f02-a215-af8715f7aff8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:48.977173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0117f5-5463-4f02-a215-af8715f7aff8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:48.977173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0117f5-5463-4f02-a215-af8715f7aff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:48.996392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-eb0117f5-5463-4f02-a215-af8715f7aff8 None None] MFA Rules not processed for user `7e04471e8c634a6586789f82204241a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:48.997587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eb0117f5-5463-4f02-a215-af8715f7aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:49.052383 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1219/2465] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:49.386125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66f31-0f45-4c18-8d36-053d8ae759cf None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:49.386650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66f31-0f45-4c18-8d36-053d8ae759cf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:49.386956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66f31-0f45-4c18-8d36-053d8ae759cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:49.411762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d5b66f31-0f45-4c18-8d36-053d8ae759cf None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:49.413134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d5b66f31-0f45-4c18-8d36-053d8ae759cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:49.475679 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1247/2466] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:38:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:49.489393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0117f5-5463-4f02-a215-af8715f7aff8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:38:49.491012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eb0117f5-5463-4f02-a215-af8715f7aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:49.569359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0117f5-5463-4f02-a215-af8715f7aff8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:38:49.576237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:38:49.577006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:49.577385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:49.577793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:49.587453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:38:49.589065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:38:49.598117 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1220/2467] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:38:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:38:58.998163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751f07-3acb-4bb5-81a0-0712a0a0b0f2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:38:58.999059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751f07-3acb-4bb5-81a0-0712a0a0b0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:38:58.999306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751f07-3acb-4bb5-81a0-0712a0a0b0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:38:59.029046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-22751f07-3acb-4bb5-81a0-0712a0a0b0f2 None None] MFA Rules not processed for user `e32778edb0e943f48e5f40453c97322b`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:38:59.030571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-22751f07-3acb-4bb5-81a0-0712a0a0b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:38:59.104511 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1248/2468] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:38:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:38:59.594154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=70 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:04.305363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:04.307182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:04.385463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef410-975e-41d1-b468-93ea7685aefd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:04.386082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef410-975e-41d1-b468-93ea7685aefd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:04.386082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef410-975e-41d1-b468-93ea7685aefd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:04.387375 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1249/2469] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 30 09:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:39:04.403012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7ac31-6f79-4318-9fd1-81753b7bec84 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:04.403297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7ac31-6f79-4318-9fd1-81753b7bec84 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:04.403569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7ac31-6f79-4318-9fd1-81753b7bec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:04.453719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-07d7ac31-6f79-4318-9fd1-81753b7bec84 None None] MFA Rules not processed for user `5989c8a1e28a43d3bc16970850ce49f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:39:04.457602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-07d7ac31-6f79-4318-9fd1-81753b7bec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:04.458782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb6d78-b277-4a2c-94d6-9b9222ac5abb service swift] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:04.468208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa20fc-e187-428e-8317-a1e086dcc7f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:04.469013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa20fc-e187-428e-8317-a1e086dcc7f0 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:04.469686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa20fc-e187-428e-8317-a1e086dcc7f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:04.470072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa20fc-e187-428e-8317-a1e086dcc7f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:04.471060 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1250/2470] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 30 09:39:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:39:04.475110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f3aa20fc-e187-428e-8317-a1e086dcc7f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:04.479342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7ac31-6f79-4318-9fd1-81753b7bec84 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:04.489590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa20fc-e187-428e-8317-a1e086dcc7f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:04.491362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa20fc-e187-428e-8317-a1e086dcc7f0 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:04.492834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7ac31-6f79-4318-9fd1-81753b7bec84 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:04.501747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ad018-7ed7-4eda-809a-fe686523461f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:04.502789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ad018-7ed7-4eda-809a-fe686523461f service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:04.503135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ad018-7ed7-4eda-809a-fe686523461f service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:04.503482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ad018-7ed7-4eda-809a-fe686523461f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:04.517697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ad018-7ed7-4eda-809a-fe686523461f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:04.518634 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1221/2471] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:39:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:04.522318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ad018-7ed7-4eda-809a-fe686523461f service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:04.555378 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1251/2472] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:39:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:39:11.914209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8d9f-8cae-4ddb-88d8-f8807f1e2db2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:11.914749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8d9f-8cae-4ddb-88d8-f8807f1e2db2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:11.914918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8d9f-8cae-4ddb-88d8-f8807f1e2db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:11.935773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-baef8d9f-8cae-4ddb-88d8-f8807f1e2db2 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:39:11.937579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-baef8d9f-8cae-4ddb-88d8-f8807f1e2db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:11.944955 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1222/2473] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 30 09:39:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:39:11.959482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ad018-7ed7-4eda-809a-fe686523461f service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:11.965665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:11.973212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ad018-7ed7-4eda-809a-fe686523461f service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:11.978716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5aa62b-ced0-48e5-920a-5dd3d5551acc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:11.979404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5aa62b-ced0-48e5-920a-5dd3d5551acc service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:11.979864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5aa62b-ced0-48e5-920a-5dd3d5551acc service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:11.980135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5aa62b-ced0-48e5-920a-5dd3d5551acc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:11.991435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5aa62b-ced0-48e5-920a-5dd3d5551acc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:11.993034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5aa62b-ced0-48e5-920a-5dd3d5551acc service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:12.004817 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1252/2474] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:39:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:39:14.031875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef8d9f-8cae-4ddb-88d8-f8807f1e2db2 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:14.034165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-baef8d9f-8cae-4ddb-88d8-f8807f1e2db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:14.112869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef8d9f-8cae-4ddb-88d8-f8807f1e2db2 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:14.117470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:14.118138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:14.118423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:14.118690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] PATH_INFO: `/v3/users/ef3be0cdb77b44d09443f3ecf8e63b79` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:14.119833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3be0cdb77b44d09443f3ecf8e63b79)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:14.121807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:14.208301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3be0cdb77b44d09443f3ecf8e63b79'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.222589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3be0cdb77b44d09443f3ecf8e63b79'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.224202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3be0cdb77b44d09443f3ecf8e63b79'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.231679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3be0cdb77b44d09443f3ecf8e63b79'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.243806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3be0cdb77b44d09443f3ecf8e63b79'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.274886 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1223/2475] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:39:14 2022] DELETE /identity/v3/users/ef3be0cdb77b44d09443f3ecf8e63b79 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:39:14.627814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5aa62b-ced0-48e5-920a-5dd3d5551acc service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:14.639042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5aa62b-ced0-48e5-920a-5dd3d5551acc service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:14.696801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:14.697423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:14.697743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:14.698014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] PATH_INFO: `/v3/projects/a530d9ebb5c749d2adaf3c428c5a4189` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:14.699170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a530d9ebb5c749d2adaf3c428c5a4189)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:14.701063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:14.747900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a530d9ebb5c749d2adaf3c428c5a4189', 'request_id': 'req-a62b833d-1db3-413f-9051-01e2c5451c55'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.756593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a530d9ebb5c749d2adaf3c428c5a4189', 'request_id': 'req-a62b833d-1db3-413f-9051-01e2c5451c55'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.764259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a530d9ebb5c749d2adaf3c428c5a4189', 'request_id': 'req-a62b833d-1db3-413f-9051-01e2c5451c55'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.776453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] The token cache is being invalidate because project a530d9ebb5c749d2adaf3c428c5a4189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:39:14.777213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.778344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a62b833d-1db3-413f-9051-01e2c5451c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:39:14.780074 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1253/2476] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:39:14 2022] DELETE /identity/v3/projects/a530d9ebb5c749d2adaf3c428c5a4189 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:39:24.268065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:24.772987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:27.131373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:27.132776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:27.191309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f9895-23a3-4084-b6a6-139ef056ec8a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:27.241895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:27.242512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:27.242794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:27.243069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:27.244815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:27.310513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:27.312008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:27.321594 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1224/2477] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:39:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:27.967838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abc805-3980-476f-8ed1-e987b611bb74 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:27.968538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abc805-3980-476f-8ed1-e987b611bb74 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:27.968538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abc805-3980-476f-8ed1-e987b611bb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:27.988501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-39abc805-3980-476f-8ed1-e987b611bb74 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:39:27.989930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-39abc805-3980-476f-8ed1-e987b611bb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:28.060777 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1254/2478] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:39:28.075637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:28.077215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:28.155318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40a4ff-cc64-4c8c-9f56-1c70f9c34759 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:28.160407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659155f-7d38-4510-a1ce-fb97deb38a5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:28.161015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659155f-7d38-4510-a1ce-fb97deb38a5b service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:28.161296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659155f-7d38-4510-a1ce-fb97deb38a5b service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:28.161567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659155f-7d38-4510-a1ce-fb97deb38a5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:28.168206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:28.172928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659155f-7d38-4510-a1ce-fb97deb38a5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:28.174589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659155f-7d38-4510-a1ce-fb97deb38a5b service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:28.182866 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1225/2479] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:39:38.045107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=25 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:38.179504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:44.899488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488281a7-fb30-4fb1-82bc-7895478e93ab None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:44.900102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488281a7-fb30-4fb1-82bc-7895478e93ab None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:44.900327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488281a7-fb30-4fb1-82bc-7895478e93ab None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:44.901737 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1255/2480] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 30 09:39:44.907781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14d36-4da6-499a-87e7-987ad4417cb6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:44.908259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14d36-4da6-499a-87e7-987ad4417cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:44.908363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14d36-4da6-499a-87e7-987ad4417cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:44.928528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fdf14d36-4da6-499a-87e7-987ad4417cb6 None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:39:44.929787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fdf14d36-4da6-499a-87e7-987ad4417cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:44.998901 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1226/2481] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:39:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:39:45.011360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488281a7-fb30-4fb1-82bc-7895478e93ab None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:45.013146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-488281a7-fb30-4fb1-82bc-7895478e93ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:45.072096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488281a7-fb30-4fb1-82bc-7895478e93ab None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:45.075514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a644f0b-b1a4-48c1-a484-df6fca9e7ddb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fe7a090bd7941bb827d301aa4949857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:45.076111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644f0b-b1a4-48c1-a484-df6fca9e7ddb service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:45.076390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644f0b-b1a4-48c1-a484-df6fca9e7ddb service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:45.076660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a644f0b-b1a4-48c1-a484-df6fca9e7ddb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:45.086696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a644f0b-b1a4-48c1-a484-df6fca9e7ddb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:45.088111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a644f0b-b1a4-48c1-a484-df6fca9e7ddb service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:45.098565 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1256/2482] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:39:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:45.177047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04990d-81ac-48b0-a0a8-cd68cc01d6f9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:45.177332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04990d-81ac-48b0-a0a8-cd68cc01d6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:45.177605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04990d-81ac-48b0-a0a8-cd68cc01d6f9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:45.178974 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1227/2483] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:39:45.183649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd50c-02c3-4a44-b62c-dd740848e5ac None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:45.183959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd50c-02c3-4a44-b62c-dd740848e5ac None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:45.184230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd50c-02c3-4a44-b62c-dd740848e5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:45.205598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bc7cd50c-02c3-4a44-b62c-dd740848e5ac None None] MFA Rules not processed for user `f41e236339784d169258f06f1090b7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:39:45.206822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bc7cd50c-02c3-4a44-b62c-dd740848e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:45.213122 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1257/2484] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:39:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:39:45.224894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04990d-81ac-48b0-a0a8-cd68cc01d6f9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:45.226607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0c04990d-81ac-48b0-a0a8-cd68cc01d6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:45.240037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04990d-81ac-48b0-a0a8-cd68cc01d6f9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:45.243803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e6230-8ce8-42da-b5b1-d3886266cc1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:45.244649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6230-8ce8-42da-b5b1-d3886266cc1a service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:45.245021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6230-8ce8-42da-b5b1-d3886266cc1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:45.245388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6230-8ce8-42da-b5b1-d3886266cc1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:45.254035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6230-8ce8-42da-b5b1-d3886266cc1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:45.255549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6230-8ce8-42da-b5b1-d3886266cc1a service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:45.263248 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1228/2485] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:39:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:45.344207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c8ef-83f8-4165-b786-3261c02c5296 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:45.344485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c8ef-83f8-4165-b786-3261c02c5296 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:45.344757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c8ef-83f8-4165-b786-3261c02c5296 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:45.346141 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1258/2486] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:39:45.351552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b377f77-9279-4133-a58e-6f39c1eae966 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:45.351839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b377f77-9279-4133-a58e-6f39c1eae966 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:45.352120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b377f77-9279-4133-a58e-6f39c1eae966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:45.372404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5b377f77-9279-4133-a58e-6f39c1eae966 None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:39:45.373671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5b377f77-9279-4133-a58e-6f39c1eae966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:45.383147 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1229/2487] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:39:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:39:45.393523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c8ef-83f8-4165-b786-3261c02c5296 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:45.404770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c8ef-83f8-4165-b786-3261c02c5296 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:45.409855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8641e1-a33a-4c65-883b-639abddebd9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:45.410468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8641e1-a33a-4c65-883b-639abddebd9b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:45.410744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8641e1-a33a-4c65-883b-639abddebd9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:45.411012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8641e1-a33a-4c65-883b-639abddebd9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:45.421971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8641e1-a33a-4c65-883b-639abddebd9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:45.423414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8641e1-a33a-4c65-883b-639abddebd9b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:45.433910 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1259/2488] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:39:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:39:47.918315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c30e49-5bd2-49df-9c89-ad6ee8b8fd5b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:47.918784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c30e49-5bd2-49df-9c89-ad6ee8b8fd5b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:47.918960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c30e49-5bd2-49df-9c89-ad6ee8b8fd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:47.939141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-03c30e49-5bd2-49df-9c89-ad6ee8b8fd5b None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:39:47.940324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-03c30e49-5bd2-49df-9c89-ad6ee8b8fd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:39:47.947328 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1230/2489] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:39:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:39:47.959132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8641e1-a33a-4c65-883b-639abddebd9b service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:39:47.970339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8641e1-a33a-4c65-883b-639abddebd9b service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:39:47.975224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72decbd0-6088-4456-80d7-0c37fa5e39cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:39:47.975808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72decbd0-6088-4456-80d7-0c37fa5e39cb service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:39:47.976088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72decbd0-6088-4456-80d7-0c37fa5e39cb service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:39:47.976357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72decbd0-6088-4456-80d7-0c37fa5e39cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:39:47.987130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72decbd0-6088-4456-80d7-0c37fa5e39cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:39:47.991462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72decbd0-6088-4456-80d7-0c37fa5e39cb service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:39:48.001476 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1260/2490] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:39:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:39:57.933180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=43 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:39:57.997506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=41 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:40:13.417286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090aa30d-fa85-4d0b-878e-979972620299 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:40:13.417793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090aa30d-fa85-4d0b-878e-979972620299 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:40:13.417884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090aa30d-fa85-4d0b-878e-979972620299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:40:13.437141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-090aa30d-fa85-4d0b-878e-979972620299 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:40:13.438449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-090aa30d-fa85-4d0b-878e-979972620299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:40:13.445743 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1231/2491] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 30 09:40:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:40:13.468407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72decbd0-6088-4456-80d7-0c37fa5e39cb service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:40:13.479577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72decbd0-6088-4456-80d7-0c37fa5e39cb service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:40:13.484408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:40:13.484989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:40:13.485491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:40:13.485810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:40:13.493570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:40:13.495157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:40:13.503863 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1261/2492] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:40:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:40:23.432187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:40:23.499777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:41:11.236556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090aa30d-fa85-4d0b-878e-979972620299 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:11.237972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-090aa30d-fa85-4d0b-878e-979972620299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:11.293614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090aa30d-fa85-4d0b-878e-979972620299 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:11.297141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:11.297779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:11.298063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:11.298341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:11.300031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:11.313100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:11.314554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:11.325394 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1232/2493] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:13.902093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:13.903525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:13.915616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6233a-2457-4707-9ccd-b4602d8abd8c service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:13.919170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:13.919757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:13.920039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:13.920313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:13.922034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:13.935272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:13.936718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:13.948147 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1262/2494] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:21.320553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=26 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:41:23.680912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:23.682317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:23.696606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48df1fb-62ee-410f-b32d-d694d404de89 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:23.702510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:23.703117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:23.703541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:23.703672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:23.705376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:23.767952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:23.769561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:23.779477 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1233/2495] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:23.942684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:41:33.775132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=28 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:41:33.949278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:33.953118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:33.965069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058df003-6653-453e-9d1d-acda1f2da48b service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:33.968687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:33.969277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:33.969559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:33.969863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:33.971600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:33.984381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:33.985848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:33.994920 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1263/2496] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:34.339280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:34.340600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:34.355482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db310eeb-1785-407a-9c9e-268459cebe19 service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:34.360995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594444f-0cb3-4c02-8524-3f31468086d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:34.361590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594444f-0cb3-4c02-8524-3f31468086d8 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:34.361902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594444f-0cb3-4c02-8524-3f31468086d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:34.362181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594444f-0cb3-4c02-8524-3f31468086d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:34.363856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c594444f-0cb3-4c02-8524-3f31468086d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:34.422226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594444f-0cb3-4c02-8524-3f31468086d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:34.423817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594444f-0cb3-4c02-8524-3f31468086d8 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:34.432945 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1234/2497] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:36.157913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:36.173457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6e55e-e27a-413a-9e50-664cb69bcbcc service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:36.179277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:36.179887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:36.180151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:36.180419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:36.182176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:36.202642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:36.204275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:36.217414 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1264/2498] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:37.457533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195a89-fff1-434e-8371-b0f31677d0cf None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:37.458703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195a89-fff1-434e-8371-b0f31677d0cf None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:37.458981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195a89-fff1-434e-8371-b0f31677d0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:37.479662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a2195a89-fff1-434e-8371-b0f31677d0cf None None] MFA Rules not processed for user `7e04471e8c634a6586789f82204241a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:41:37.480869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a2195a89-fff1-434e-8371-b0f31677d0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:37.534443 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1235/2499] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:41:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:41:41.397947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:41.413179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf4561-a9be-45f1-ac34-f5bc8f04a32b service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:41.420468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:41.421284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:41.421670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:41.422001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:41.423839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:41.436987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:41.438715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:41.453951 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1265/2500] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:42.086135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e459f9e-3d8c-4073-8fb4-6361ac9894bf None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:42.086654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e459f9e-3d8c-4073-8fb4-6361ac9894bf None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:42.086764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e459f9e-3d8c-4073-8fb4-6361ac9894bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:42.107607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9e459f9e-3d8c-4073-8fb4-6361ac9894bf None None] MFA Rules not processed for user `e32778edb0e943f48e5f40453c97322b`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:41:42.108910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9e459f9e-3d8c-4073-8fb4-6361ac9894bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:42.121205 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1236/2501] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:41:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:41:42.440153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:42.450156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03787367-3692-4733-a64a-ffc389f553d3 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:42.453834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb655c-bcf3-4ccf-8427-fe1d583467d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:42.454426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb655c-bcf3-4ccf-8427-fe1d583467d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:42.454702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb655c-bcf3-4ccf-8427-fe1d583467d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:42.454969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb655c-bcf3-4ccf-8427-fe1d583467d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:42.456828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-98cb655c-bcf3-4ccf-8427-fe1d583467d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:42.467203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb655c-bcf3-4ccf-8427-fe1d583467d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:42.468634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb655c-bcf3-4ccf-8427-fe1d583467d3 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:42.477012 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1266/2502] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:42.829221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e459f9e-3d8c-4073-8fb4-6361ac9894bf None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:42.830638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9e459f9e-3d8c-4073-8fb4-6361ac9894bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:43.032396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e459f9e-3d8c-4073-8fb4-6361ac9894bf None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:43.038726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:43.039457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:43.039674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:43.040036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:43.041971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:43.056247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:43.057938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:43.069113 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1237/2503] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:46.081011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c49e39-a2fb-4fa6-9b0c-6faa4b661820 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:46.081527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c49e39-a2fb-4fa6-9b0c-6faa4b661820 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:46.081622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c49e39-a2fb-4fa6-9b0c-6faa4b661820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:46.101981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-28c49e39-a2fb-4fa6-9b0c-6faa4b661820 None None] MFA Rules not processed for user `8efb5824b0c74435b41f3d7911c20874`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:41:46.103422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-28c49e39-a2fb-4fa6-9b0c-6faa4b661820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:46.111515 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1267/2504] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:41:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:41:46.700660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:46.711546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d3784-d15f-4892-82b8-441146480b84 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:46.715395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:46.716112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:46.716519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:46.716876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:46.718987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:46.729244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:46.730816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:46.739952 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1238/2505] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:52.881176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c49e39-a2fb-4fa6-9b0c-6faa4b661820 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:41:52.896909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c49e39-a2fb-4fa6-9b0c-6faa4b661820 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:41:52.904192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea4f6d-5e53-4b92-8e1b-a9bfb8838c67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:41:52.904805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4f6d-5e53-4b92-8e1b-a9bfb8838c67 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:41:52.905080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4f6d-5e53-4b92-8e1b-a9bfb8838c67 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:41:52.905344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4f6d-5e53-4b92-8e1b-a9bfb8838c67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:41:52.907406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e2ea4f6d-5e53-4b92-8e1b-a9bfb8838c67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:41:52.922113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea4f6d-5e53-4b92-8e1b-a9bfb8838c67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:41:52.924072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea4f6d-5e53-4b92-8e1b-a9bfb8838c67 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:41:52.942238 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1268/2506] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:41:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:41:56.735920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:42:02.931004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=33 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:42:47.498103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:42:47.507965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1879f-b8a4-420f-9458-c2afe957b13c service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:42:47.511494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c793a5c-dc0e-48d0-8d46-a9e7f5a1e03a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:42:47.512109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c793a5c-dc0e-48d0-8d46-a9e7f5a1e03a service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:42:47.512387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c793a5c-dc0e-48d0-8d46-a9e7f5a1e03a service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:42:47.512657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c793a5c-dc0e-48d0-8d46-a9e7f5a1e03a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:42:47.514438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5c793a5c-dc0e-48d0-8d46-a9e7f5a1e03a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:42:47.525900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c793a5c-dc0e-48d0-8d46-a9e7f5a1e03a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:42:47.527292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c793a5c-dc0e-48d0-8d46-a9e7f5a1e03a service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:42:47.535982 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1239/2507] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:42:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:42:56.688003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d141bd-74f1-442a-9518-4424f870b4b6 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:42:56.688600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d141bd-74f1-442a-9518-4424f870b4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:42:56.688919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d141bd-74f1-442a-9518-4424f870b4b6 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:42:56.690685 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1269/2508] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:42:56.697871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d325-4cf9-4dc4-9998-f52db6ce964e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:42:56.698384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d325-4cf9-4dc4-9998-f52db6ce964e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:42:56.698571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d325-4cf9-4dc4-9998-f52db6ce964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:42:56.718815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7cb2d325-4cf9-4dc4-9998-f52db6ce964e None None] MFA Rules not processed for user `f41e236339784d169258f06f1090b7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:42:56.720277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7cb2d325-4cf9-4dc4-9998-f52db6ce964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:42:56.727544 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1240/2509] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:42:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:42:56.742124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d141bd-74f1-442a-9518-4424f870b4b6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:42:56.754080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d141bd-74f1-442a-9518-4424f870b4b6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:42:56.758269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5ad64-c560-49b4-8a4e-8d42b580d12e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:42:56.759056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5ad64-c560-49b4-8a4e-8d42b580d12e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:42:56.759463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5ad64-c560-49b4-8a4e-8d42b580d12e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:42:56.759742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5ad64-c560-49b4-8a4e-8d42b580d12e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:42:56.767963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5ad64-c560-49b4-8a4e-8d42b580d12e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:42:56.770228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5ad64-c560-49b4-8a4e-8d42b580d12e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:42:56.779053 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1270/2510] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:42:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:42:56.883630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eee122-0de7-420f-ad56-ee763c1f8cc4 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:42:56.883980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eee122-0de7-420f-ad56-ee763c1f8cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:42:56.884314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eee122-0de7-420f-ad56-ee763c1f8cc4 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:42:56.885724 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1241/2511] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:42:56.891725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c92c5d-dc02-457d-b87f-b2c737aa6616 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:42:56.892041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c92c5d-dc02-457d-b87f-b2c737aa6616 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:42:56.892398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c92c5d-dc02-457d-b87f-b2c737aa6616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:42:56.913580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a3c92c5d-dc02-457d-b87f-b2c737aa6616 None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:42:56.914984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a3c92c5d-dc02-457d-b87f-b2c737aa6616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:42:56.926267 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1271/2512] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:42:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:42:56.938454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eee122-0de7-420f-ad56-ee763c1f8cc4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:42:56.951544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eee122-0de7-420f-ad56-ee763c1f8cc4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:42:56.955397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:42:56.956052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:42:56.956326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:42:56.956594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:42:56.967861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:42:56.969478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:42:56.981697 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1242/2513] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:42:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:43:06.907939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=9 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:06.976501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=15 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:12.401202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c92c5d-dc02-457d-b87f-b2c737aa6616 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:12.410839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c92c5d-dc02-457d-b87f-b2c737aa6616 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:12.414310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:12.414900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:12.415170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:12.415435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:12.417120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:12.479049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:12.480479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:12.490423 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1272/2514] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:15.926816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:15.928248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:15.943681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c9da4-3855-49ab-b607-69d637b76c30 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:15.948592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:15.949208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:15.949469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:15.949775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:15.951513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:15.963932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:15.965497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:15.980485 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1243/2515] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:15.983174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:15.984672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:15.997825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57bc92-1295-41b6-95fd-0f15b1f70d99 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:16.002374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:16.003007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.003282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.003556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] PATH_INFO: `/v3/users/7e04471e8c634a6586789f82204241a7` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.005617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e04471e8c634a6586789f82204241a7)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:16.007851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:16.054515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e04471e8c634a6586789f82204241a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.061417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e04471e8c634a6586789f82204241a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.062170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e04471e8c634a6586789f82204241a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.062838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e04471e8c634a6586789f82204241a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.070787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-29140005-8397-49c7-ae27-24efbf1510cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e04471e8c634a6586789f82204241a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.088440 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1273/2516] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:15 2022] DELETE /identity/v3/users/7e04471e8c634a6586789f82204241a7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:16.258641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:16.271171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bfe4c-38f9-4e56-b600-210cc7697311 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:16.327560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:16.328190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.328473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.328747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] PATH_INFO: `/v3/projects/4290a542e73045309f6ceb8d4f5a267b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.329887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=4290a542e73045309f6ceb8d4f5a267b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:16.331781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:16.376772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4290a542e73045309f6ceb8d4f5a267b', 'request_id': 'req-af2e64fa-b33e-497d-a609-f2c4154073ce'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.384936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4290a542e73045309f6ceb8d4f5a267b', 'request_id': 'req-af2e64fa-b33e-497d-a609-f2c4154073ce'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.391341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4290a542e73045309f6ceb8d4f5a267b', 'request_id': 'req-af2e64fa-b33e-497d-a609-f2c4154073ce'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.401966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] The token cache is being invalidate because project 4290a542e73045309f6ceb8d4f5a267b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:16.402260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.402915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e64fa-b33e-497d-a609-f2c4154073ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:16.404405 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1244/2517] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:16 2022] DELETE /identity/v3/projects/4290a542e73045309f6ceb8d4f5a267b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:16.409348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ffb1-e60e-402e-b847-74fc789864fe None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.409648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ffb1-e60e-402e-b847-74fc789864fe None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.409948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6ffb1-e60e-402e-b847-74fc789864fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.429881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-adf6ffb1-e60e-402e-b847-74fc789864fe None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:16.431077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-adf6ffb1-e60e-402e-b847-74fc789864fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:16.490219 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1274/2518] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:16.497720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09456b5-edc3-4f57-a4b2-a8651439c2a5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.498008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09456b5-edc3-4f57-a4b2-a8651439c2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.498284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09456b5-edc3-4f57-a4b2-a8651439c2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.517743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f09456b5-edc3-4f57-a4b2-a8651439c2a5 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:16.518945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f09456b5-edc3-4f57-a4b2-a8651439c2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:16.530153 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1245/2519] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:43:16.534423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6ffb1-e60e-402e-b847-74fc789864fe None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:16.545096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6ffb1-e60e-402e-b847-74fc789864fe None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:16.549590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:16.550204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.550476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.550813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.551980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:43:16.552287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:16.553921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:16.555400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:16.567821 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1275/2520] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:43:16.572173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09456b5-edc3-4f57-a4b2-a8651439c2a5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:16.581128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09456b5-edc3-4f57-a4b2-a8651439c2a5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:16.585769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5345b5-c3e4-4d14-bfad-51b9d804607d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:16.586361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5345b5-c3e4-4d14-bfad-51b9d804607d admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.586636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5345b5-c3e4-4d14-bfad-51b9d804607d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.586905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5345b5-c3e4-4d14-bfad-51b9d804607d admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.587525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5345b5-c3e4-4d14-bfad-51b9d804607d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:16.589406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5345b5-c3e4-4d14-bfad-51b9d804607d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:16.622726 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1246/2521] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:16 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:16.627130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:16.637049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bcf07-acc1-4a06-ad23-89fcb3854431 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:16.692343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9f311-c5e6-40ab-b81a-c37a42511b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:16.692917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f311-c5e6-40ab-b81a-c37a42511b3f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.693196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f311-c5e6-40ab-b81a-c37a42511b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.693494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9f311-c5e6-40ab-b81a-c37a42511b3f admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.694148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9f311-c5e6-40ab-b81a-c37a42511b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:16.696056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9f311-c5e6-40ab-b81a-c37a42511b3f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:16.716584 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1276/2522] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:16 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:16.720716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5345b5-c3e4-4d14-bfad-51b9d804607d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:16.731317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5345b5-c3e4-4d14-bfad-51b9d804607d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:16.735904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32de52f-95f9-429c-8e73-5a5df2608943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:16.736494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de52f-95f9-429c-8e73-5a5df2608943 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.736774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de52f-95f9-429c-8e73-5a5df2608943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.737047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de52f-95f9-429c-8e73-5a5df2608943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.737715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32de52f-95f9-429c-8e73-5a5df2608943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:16.739924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32de52f-95f9-429c-8e73-5a5df2608943 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:16.749588 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1247/2523] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:16.753260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9f311-c5e6-40ab-b81a-c37a42511b3f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:16.763380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9f311-c5e6-40ab-b81a-c37a42511b3f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:16.768257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c69d7-639f-4566-ab24-3534940ec8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:16.768841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c69d7-639f-4566-ab24-3534940ec8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.769124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c69d7-639f-4566-ab24-3534940ec8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.769400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c69d7-639f-4566-ab24-3534940ec8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.770161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c69d7-639f-4566-ab24-3534940ec8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:16.771764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c69d7-639f-4566-ab24-3534940ec8b4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:16.782062 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1277/2524] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:16.786575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32de52f-95f9-429c-8e73-5a5df2608943 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:16.796787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32de52f-95f9-429c-8e73-5a5df2608943 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:16.801979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac537600-84e8-464f-b1de-8aa7e798b93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:16.802621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac537600-84e8-464f-b1de-8aa7e798b93e admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.802902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac537600-84e8-464f-b1de-8aa7e798b93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.803176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac537600-84e8-464f-b1de-8aa7e798b93e admin admin] PATH_INFO: `/v3/projects/77e6f8f64fe0408e913eed33ee35efdc/users/106350d5737343eaaa261d4f6a4757d9/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.817594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac537600-84e8-464f-b1de-8aa7e798b93e admin admin] RBAC: Authorizing `identity:create_grant(project_id=77e6f8f64fe0408e913eed33ee35efdc, user_id=106350d5737343eaaa261d4f6a4757d9, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:16.820356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac537600-84e8-464f-b1de-8aa7e798b93e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:16.832464 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1248/2525] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:16 2022] PUT /identity/v3/projects/77e6f8f64fe0408e913eed33ee35efdc/users/106350d5737343eaaa261d4f6a4757d9/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:16.843838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8693923-1c89-4edd-a61f-749b150f383b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:16.844124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8693923-1c89-4edd-a61f-749b150f383b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:16.844396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8693923-1c89-4edd-a61f-749b150f383b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:16.872190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c8693923-1c89-4edd-a61f-749b150f383b None None] MFA Rules not processed for user `106350d5737343eaaa261d4f6a4757d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:16.873396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c8693923-1c89-4edd-a61f-749b150f383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:16.941819 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1278/2526] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:16.946446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac537600-84e8-464f-b1de-8aa7e798b93e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:16.963095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:17.018942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac537600-84e8-464f-b1de-8aa7e798b93e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:17.024293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ef798-5c57-45cd-9a29-71dfd1933b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:17.024897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ef798-5c57-45cd-9a29-71dfd1933b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:17.025175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ef798-5c57-45cd-9a29-71dfd1933b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:17.025446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ef798-5c57-45cd-9a29-71dfd1933b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:17.026110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ef798-5c57-45cd-9a29-71dfd1933b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:17.027997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ef798-5c57-45cd-9a29-71dfd1933b3e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:17.058304 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1249/2527] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:16 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:17.062540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8693923-1c89-4edd-a61f-749b150f383b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:17.073178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8693923-1c89-4edd-a61f-749b150f383b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:17.128647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fe5d1-8d49-4252-84e9-24c1739e336c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:17.129464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fe5d1-8d49-4252-84e9-24c1739e336c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:17.129775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fe5d1-8d49-4252-84e9-24c1739e336c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:17.130033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fe5d1-8d49-4252-84e9-24c1739e336c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:17.130666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fe5d1-8d49-4252-84e9-24c1739e336c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:17.132565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fe5d1-8d49-4252-84e9-24c1739e336c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:17.141523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:17.154126 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1279/2528] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:17 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:17.158104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ef798-5c57-45cd-9a29-71dfd1933b3e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:17.169252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ef798-5c57-45cd-9a29-71dfd1933b3e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:17.173786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cc34d-c4e4-483e-b076-93cb038c0359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:17.174393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc34d-c4e4-483e-b076-93cb038c0359 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:17.174650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc34d-c4e4-483e-b076-93cb038c0359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:17.174918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cc34d-c4e4-483e-b076-93cb038c0359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:17.175547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969cc34d-c4e4-483e-b076-93cb038c0359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:17.177142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969cc34d-c4e4-483e-b076-93cb038c0359 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:17.186972 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1250/2529] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:17.191029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fe5d1-8d49-4252-84e9-24c1739e336c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:17.201053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fe5d1-8d49-4252-84e9-24c1739e336c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:17.205597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cf7e5-5a98-4d5e-a391-8b54ccb8febf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:17.206263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cf7e5-5a98-4d5e-a391-8b54ccb8febf admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:17.206544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cf7e5-5a98-4d5e-a391-8b54ccb8febf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:17.206816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cf7e5-5a98-4d5e-a391-8b54ccb8febf admin admin] PATH_INFO: `/v3/projects/28fed1ec17ad41a8bf0b06aaef311b6c/users/41f526abe62d47f7b122686e17b2b037/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:17.221768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cf7e5-5a98-4d5e-a391-8b54ccb8febf admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fed1ec17ad41a8bf0b06aaef311b6c, user_id=41f526abe62d47f7b122686e17b2b037, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:17.225376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cf7e5-5a98-4d5e-a391-8b54ccb8febf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:17.238681 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1280/2530] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:17 2022] PUT /identity/v3/projects/28fed1ec17ad41a8bf0b06aaef311b6c/users/41f526abe62d47f7b122686e17b2b037/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:17.242562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cc34d-c4e4-483e-b076-93cb038c0359 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:17.252472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cc34d-c4e4-483e-b076-93cb038c0359 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:17.310297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d608362-1f78-420f-bcad-bd36c57a9e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:17.310813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d608362-1f78-420f-bcad-bd36c57a9e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:17.311085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d608362-1f78-420f-bcad-bd36c57a9e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:17.311380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d608362-1f78-420f-bcad-bd36c57a9e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:17.312006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d608362-1f78-420f-bcad-bd36c57a9e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:17.313612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d608362-1f78-420f-bcad-bd36c57a9e37 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:17.323914 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1251/2531] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:17.328163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cf7e5-5a98-4d5e-a391-8b54ccb8febf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:17.338902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cf7e5-5a98-4d5e-a391-8b54ccb8febf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:17.343434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:17.344063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:17.344338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:17.344625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] PATH_INFO: `/v3/projects/28fed1ec17ad41a8bf0b06aaef311b6c/users/41f526abe62d47f7b122686e17b2b037/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:17.346550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fed1ec17ad41a8bf0b06aaef311b6c, user_id=41f526abe62d47f7b122686e17b2b037, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:17.349310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:17.359242 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1281/2532] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:17 2022] PUT /identity/v3/projects/28fed1ec17ad41a8bf0b06aaef311b6c/users/41f526abe62d47f7b122686e17b2b037/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:17.369071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa3bae-b9b5-4158-a9ca-be3ea6d6771a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:17.369350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa3bae-b9b5-4158-a9ca-be3ea6d6771a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:17.369618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa3bae-b9b5-4158-a9ca-be3ea6d6771a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:17.398598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7efa3bae-b9b5-4158-a9ca-be3ea6d6771a None None] MFA Rules not processed for user `41f526abe62d47f7b122686e17b2b037`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:17.399803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7efa3bae-b9b5-4158-a9ca-be3ea6d6771a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:17.466227 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1252/2533] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:17.477750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:17.479441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:17.535908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd52b8-b63b-41c3-95d8-e19a87d3db3a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:17.539481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac06db-07ac-424f-8749-a26f8c709728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:17.540070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac06db-07ac-424f-8749-a26f8c709728 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:17.540358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac06db-07ac-424f-8749-a26f8c709728 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:17.540609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac06db-07ac-424f-8749-a26f8c709728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:17.549534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac06db-07ac-424f-8749-a26f8c709728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:17.551520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac06db-07ac-424f-8749-a26f8c709728 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:17.561000 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1282/2534] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:17.944957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa3bae-b9b5-4158-a9ca-be3ea6d6771a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:17.954432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:18.007385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa3bae-b9b5-4158-a9ca-be3ea6d6771a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:18.011775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:18.012393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:18.012664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:18.012933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] PATH_INFO: `/v3/users/106350d5737343eaaa261d4f6a4757d9` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:18.014054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=106350d5737343eaaa261d4f6a4757d9)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:18.016216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:18.064094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106350d5737343eaaa261d4f6a4757d9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.070908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106350d5737343eaaa261d4f6a4757d9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.071564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106350d5737343eaaa261d4f6a4757d9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.072169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106350d5737343eaaa261d4f6a4757d9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.079544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106350d5737343eaaa261d4f6a4757d9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.096406 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1253/2535] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:17 2022] DELETE /identity/v3/users/106350d5737343eaaa261d4f6a4757d9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:18.101699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc4862-bb78-47a1-aa8f-043d5c0fbe22 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:18.102186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc4862-bb78-47a1-aa8f-043d5c0fbe22 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:18.102305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc4862-bb78-47a1-aa8f-043d5c0fbe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:18.122656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0ccc4862-bb78-47a1-aa8f-043d5c0fbe22 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:18.123850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0ccc4862-bb78-47a1-aa8f-043d5c0fbe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:18.180782 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1283/2536] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:18.190682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:18.192155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:18.248550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e697a-632e-4153-a11c-d3ad40499203 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:18.252136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc807ffb-0e43-491b-be29-5544b9c4d799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:18.252758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc807ffb-0e43-491b-be29-5544b9c4d799 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:18.253035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc807ffb-0e43-491b-be29-5544b9c4d799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:18.253305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc807ffb-0e43-491b-be29-5544b9c4d799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:18.262732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc807ffb-0e43-491b-be29-5544b9c4d799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:18.264141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc807ffb-0e43-491b-be29-5544b9c4d799 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:18.274311 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1254/2537] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:18.544406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc4862-bb78-47a1-aa8f-043d5c0fbe22 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:18.545843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0ccc4862-bb78-47a1-aa8f-043d5c0fbe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:18.558041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc4862-bb78-47a1-aa8f-043d5c0fbe22 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:18.562546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:18.563169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:18.563441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:18.563707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] PATH_INFO: `/v3/projects/77e6f8f64fe0408e913eed33ee35efdc` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:18.564814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] RBAC: Authorizing `identity:delete_project(project_id=77e6f8f64fe0408e913eed33ee35efdc)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:18.566907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:18.613371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77e6f8f64fe0408e913eed33ee35efdc', 'request_id': 'req-932f0d57-2322-4615-b05a-ba286396684d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.615985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:18.621571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77e6f8f64fe0408e913eed33ee35efdc', 'request_id': 'req-932f0d57-2322-4615-b05a-ba286396684d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.628493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77e6f8f64fe0408e913eed33ee35efdc', 'request_id': 'req-932f0d57-2322-4615-b05a-ba286396684d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.638581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] The token cache is being invalidate because project 77e6f8f64fe0408e913eed33ee35efdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:18.638865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.639487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.640926 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1284/2538] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:18 2022] DELETE /identity/v3/projects/77e6f8f64fe0408e913eed33ee35efdc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:18.645736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc807ffb-0e43-491b-be29-5544b9c4d799 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:18.655306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc807ffb-0e43-491b-be29-5544b9c4d799 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:18.684395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:18.715522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:18.716147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:18.716423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:18.716693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] PATH_INFO: `/v3/users/41f526abe62d47f7b122686e17b2b037` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:18.717813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f526abe62d47f7b122686e17b2b037)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:18.719716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:18.763170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f526abe62d47f7b122686e17b2b037'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.770608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f526abe62d47f7b122686e17b2b037'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.771247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f526abe62d47f7b122686e17b2b037'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.771856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f526abe62d47f7b122686e17b2b037'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.779098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd0916a4-5d39-40c9-aada-ba1ee8f929d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f526abe62d47f7b122686e17b2b037'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:18.795651 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1255/2539] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:18 2022] DELETE /identity/v3/users/41f526abe62d47f7b122686e17b2b037 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:19.061488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.062925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:19.076087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f0d57-2322-4615-b05a-ba286396684d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.133513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.134214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.134484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.134759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] PATH_INFO: `/v3/projects/28fed1ec17ad41a8bf0b06aaef311b6c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.135872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28fed1ec17ad41a8bf0b06aaef311b6c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.137896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.187412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28fed1ec17ad41a8bf0b06aaef311b6c', 'request_id': 'req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:19.194656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28fed1ec17ad41a8bf0b06aaef311b6c', 'request_id': 'req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:19.200964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28fed1ec17ad41a8bf0b06aaef311b6c', 'request_id': 'req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:19.211713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] The token cache is being invalidate because project 28fed1ec17ad41a8bf0b06aaef311b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:19.211998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:19.212630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eacb13ea-969f-4dbc-bcfe-036cdc4673c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:19.214166 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1285/2540] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:19 2022] DELETE /identity/v3/projects/28fed1ec17ad41a8bf0b06aaef311b6c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:19.219040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8669b95-b4d5-443b-92c1-482d7195c31d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.219497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8669b95-b4d5-443b-92c1-482d7195c31d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.219626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8669b95-b4d5-443b-92c1-482d7195c31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.239819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a8669b95-b4d5-443b-92c1-482d7195c31d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:19.241021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a8669b95-b4d5-443b-92c1-482d7195c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:19.298237 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1256/2541] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:19.306635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700182f9-9a69-4163-8d15-e89ddc363769 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.306899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700182f9-9a69-4163-8d15-e89ddc363769 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.307165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700182f9-9a69-4163-8d15-e89ddc363769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.326869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-700182f9-9a69-4163-8d15-e89ddc363769 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:19.328043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-700182f9-9a69-4163-8d15-e89ddc363769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:19.335971 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1286/2542] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:19.340307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8669b95-b4d5-443b-92c1-482d7195c31d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.351492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8669b95-b4d5-443b-92c1-482d7195c31d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.355995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.356590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.356864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.357130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.357798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:43:19.358095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.359669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.371727 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1257/2543] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:43:19.375968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700182f9-9a69-4163-8d15-e89ddc363769 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.386120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700182f9-9a69-4163-8d15-e89ddc363769 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.390743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4f465-603e-410c-b4de-3f5215e7dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.391590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4f465-603e-410c-b4de-3f5215e7dd28 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.391868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4f465-603e-410c-b4de-3f5215e7dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.392137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4f465-603e-410c-b4de-3f5215e7dd28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.392774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4f465-603e-410c-b4de-3f5215e7dd28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.394878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4f465-603e-410c-b4de-3f5215e7dd28 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.421688 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1287/2544] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:19 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:19.426379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.436378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf169a-d16e-4cd2-9155-f00bcef58956 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.469164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:19.495775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdab47-e061-43c3-b3d1-5502c3c0ce86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.496373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdab47-e061-43c3-b3d1-5502c3c0ce86 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.496652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdab47-e061-43c3-b3d1-5502c3c0ce86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.496921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdab47-e061-43c3-b3d1-5502c3c0ce86 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.497548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdab47-e061-43c3-b3d1-5502c3c0ce86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.499517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdab47-e061-43c3-b3d1-5502c3c0ce86 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.521607 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1258/2545] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:19 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:43:19.525946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4f465-603e-410c-b4de-3f5215e7dd28 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.536674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4f465-603e-410c-b4de-3f5215e7dd28 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.541415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5487892-bd89-47d7-8cbc-3f23e3f6a4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.542051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5487892-bd89-47d7-8cbc-3f23e3f6a4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.542332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5487892-bd89-47d7-8cbc-3f23e3f6a4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.542649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5487892-bd89-47d7-8cbc-3f23e3f6a4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.543293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5487892-bd89-47d7-8cbc-3f23e3f6a4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.544941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5487892-bd89-47d7-8cbc-3f23e3f6a4b7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.555489 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1288/2546] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:19.559261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdab47-e061-43c3-b3d1-5502c3c0ce86 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.568865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdab47-e061-43c3-b3d1-5502c3c0ce86 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.573435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3bf998-c17c-4ed0-b88f-b06ef7c6a326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.574053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf998-c17c-4ed0-b88f-b06ef7c6a326 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.574330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf998-c17c-4ed0-b88f-b06ef7c6a326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.574599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3bf998-c17c-4ed0-b88f-b06ef7c6a326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.575228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bf998-c17c-4ed0-b88f-b06ef7c6a326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.576790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3bf998-c17c-4ed0-b88f-b06ef7c6a326 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.586374 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1259/2547] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:19.590481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5487892-bd89-47d7-8cbc-3f23e3f6a4b7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.601227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5487892-bd89-47d7-8cbc-3f23e3f6a4b7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.605805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1bf8-5cfd-452f-949b-2a5185cab666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.606440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1bf8-5cfd-452f-949b-2a5185cab666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.606720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1bf8-5cfd-452f-949b-2a5185cab666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.606990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1bf8-5cfd-452f-949b-2a5185cab666 admin admin] PATH_INFO: `/v3/projects/bc85b9a668974ee0a5c84af05486e8f0/users/2467310f208a4b3e92686ad253514c4f/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.621542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1bf8-5cfd-452f-949b-2a5185cab666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc85b9a668974ee0a5c84af05486e8f0, user_id=2467310f208a4b3e92686ad253514c4f, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.624221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1bf8-5cfd-452f-949b-2a5185cab666 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.637029 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1289/2548] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:19 2022] PUT /identity/v3/projects/bc85b9a668974ee0a5c84af05486e8f0/users/2467310f208a4b3e92686ad253514c4f/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:19.648603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f4ca-766a-47df-939c-907a3fb8fbd5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.648915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f4ca-766a-47df-939c-907a3fb8fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.649190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f4ca-766a-47df-939c-907a3fb8fbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.678411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e854f4ca-766a-47df-939c-907a3fb8fbd5 None None] MFA Rules not processed for user `2467310f208a4b3e92686ad253514c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:19.679555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e854f4ca-766a-47df-939c-907a3fb8fbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:19.742832 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1260/2549] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:19.747469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1bf8-5cfd-452f-949b-2a5185cab666 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.756518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:19.810562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1bf8-5cfd-452f-949b-2a5185cab666 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.815146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1e3ee-5982-4b10-81fc-8c1eb36a04c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.815738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1e3ee-5982-4b10-81fc-8c1eb36a04c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.816017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1e3ee-5982-4b10-81fc-8c1eb36a04c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.816346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1e3ee-5982-4b10-81fc-8c1eb36a04c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.816959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1e3ee-5982-4b10-81fc-8c1eb36a04c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.818896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1e3ee-5982-4b10-81fc-8c1eb36a04c3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.844440 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1290/2550] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:19 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:19.849067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f4ca-766a-47df-939c-907a3fb8fbd5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.859500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f4ca-766a-47df-939c-907a3fb8fbd5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.914813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439ab35-252f-4938-8a9e-3d796e8ede24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.915413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439ab35-252f-4938-8a9e-3d796e8ede24 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.915695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439ab35-252f-4938-8a9e-3d796e8ede24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.915957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439ab35-252f-4938-8a9e-3d796e8ede24 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.916567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439ab35-252f-4938-8a9e-3d796e8ede24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.918496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439ab35-252f-4938-8a9e-3d796e8ede24 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.939684 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1261/2551] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:19 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:19.943628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1e3ee-5982-4b10-81fc-8c1eb36a04c3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.953835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1e3ee-5982-4b10-81fc-8c1eb36a04c3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.958893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe380d-a384-4306-a6f3-f599edc12278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.959639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe380d-a384-4306-a6f3-f599edc12278 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.959954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe380d-a384-4306-a6f3-f599edc12278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.960273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe380d-a384-4306-a6f3-f599edc12278 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:19.960949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe380d-a384-4306-a6f3-f599edc12278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:19.962853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe380d-a384-4306-a6f3-f599edc12278 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:19.973997 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1291/2552] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:19.978059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439ab35-252f-4938-8a9e-3d796e8ede24 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:19.988097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439ab35-252f-4938-8a9e-3d796e8ede24 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:19.993225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b9550-c64f-460e-bcc1-58e06d3af8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:19.993900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b9550-c64f-460e-bcc1-58e06d3af8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:19.994180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b9550-c64f-460e-bcc1-58e06d3af8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:19.994453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b9550-c64f-460e-bcc1-58e06d3af8d6 admin admin] PATH_INFO: `/v3/projects/4a8f63f0a5b3400797291f7be59894fe/users/85121f63338e438e8d061567dd5d1789/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:20.009334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b9550-c64f-460e-bcc1-58e06d3af8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8f63f0a5b3400797291f7be59894fe, user_id=85121f63338e438e8d061567dd5d1789, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:20.011999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b9550-c64f-460e-bcc1-58e06d3af8d6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:20.025902 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1262/2553] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:19 2022] PUT /identity/v3/projects/4a8f63f0a5b3400797291f7be59894fe/users/85121f63338e438e8d061567dd5d1789/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:20.029764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe380d-a384-4306-a6f3-f599edc12278 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:20.040460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe380d-a384-4306-a6f3-f599edc12278 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:20.097395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d14b4-9ec7-481f-9b24-1e179b557c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:20.098835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d14b4-9ec7-481f-9b24-1e179b557c2f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:20.099115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d14b4-9ec7-481f-9b24-1e179b557c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:20.099383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d14b4-9ec7-481f-9b24-1e179b557c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:20.100014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d14b4-9ec7-481f-9b24-1e179b557c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:20.101606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d14b4-9ec7-481f-9b24-1e179b557c2f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:20.111110 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1292/2554] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:20.116792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b9550-c64f-460e-bcc1-58e06d3af8d6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:20.127930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b9550-c64f-460e-bcc1-58e06d3af8d6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:20.132563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:20.133215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:20.133495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:20.133799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] PATH_INFO: `/v3/projects/4a8f63f0a5b3400797291f7be59894fe/users/85121f63338e438e8d061567dd5d1789/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:20.135722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8f63f0a5b3400797291f7be59894fe, user_id=85121f63338e438e8d061567dd5d1789, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:20.138405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:20.144110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:20.150413 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1263/2555] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:20 2022] PUT /identity/v3/projects/4a8f63f0a5b3400797291f7be59894fe/users/85121f63338e438e8d061567dd5d1789/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:20.161172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb65e-82ab-4ff5-bef8-2f58233e6ef0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:20.161708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb65e-82ab-4ff5-bef8-2f58233e6ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:20.161836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb65e-82ab-4ff5-bef8-2f58233e6ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:20.188863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9f1eb65e-82ab-4ff5-bef8-2f58233e6ef0 None None] MFA Rules not processed for user `85121f63338e438e8d061567dd5d1789`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:20.190157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9f1eb65e-82ab-4ff5-bef8-2f58233e6ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:20.258593 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1293/2556] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:20.270565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:20.272372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:20.328660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52995090-3d0d-48ec-85cf-917d99b9d396 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:20.332965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251caf0a-deb0-4f8e-b862-82b765257a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:20.333598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251caf0a-deb0-4f8e-b862-82b765257a77 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:20.333910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251caf0a-deb0-4f8e-b862-82b765257a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:20.334179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251caf0a-deb0-4f8e-b862-82b765257a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:20.342995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251caf0a-deb0-4f8e-b862-82b765257a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:20.344407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251caf0a-deb0-4f8e-b862-82b765257a77 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:20.353617 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1264/2557] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:20.429165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb65e-82ab-4ff5-bef8-2f58233e6ef0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:20.433722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:20.439914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb65e-82ab-4ff5-bef8-2f58233e6ef0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:20.444272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22806e9a-5352-4af4-aa4c-56136b7da0b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:20.444810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806e9a-5352-4af4-aa4c-56136b7da0b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:20.445087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806e9a-5352-4af4-aa4c-56136b7da0b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:20.445354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806e9a-5352-4af4-aa4c-56136b7da0b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:20.501250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22806e9a-5352-4af4-aa4c-56136b7da0b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:20.502728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22806e9a-5352-4af4-aa4c-56136b7da0b7 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:20.512008 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1294/2558] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:23.224905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251caf0a-deb0-4f8e-b862-82b765257a77 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:23.285511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251caf0a-deb0-4f8e-b862-82b765257a77 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:23.290051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:23.290673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:23.290952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:23.291227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] PATH_INFO: `/v3/users/2467310f208a4b3e92686ad253514c4f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:23.292330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2467310f208a4b3e92686ad253514c4f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:23.294286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:23.339787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2467310f208a4b3e92686ad253514c4f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.346633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2467310f208a4b3e92686ad253514c4f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.347338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2467310f208a4b3e92686ad253514c4f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.348103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2467310f208a4b3e92686ad253514c4f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.355827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2467310f208a4b3e92686ad253514c4f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.373483 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1265/2559] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:23 2022] DELETE /identity/v3/users/2467310f208a4b3e92686ad253514c4f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:23.378157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd43aa-e96e-49d3-86a4-70e2b739f09e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:23.379155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd43aa-e96e-49d3-86a4-70e2b739f09e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:23.379332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd43aa-e96e-49d3-86a4-70e2b739f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:23.399287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7bcd43aa-e96e-49d3-86a4-70e2b739f09e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:23.400486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7bcd43aa-e96e-49d3-86a4-70e2b739f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:23.458171 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1295/2560] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:23.467750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:23.469138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:23.490907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:23.527435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932a224-401d-4abc-a797-776980a9c61d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:23.531020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25cc1d-7ddf-49a1-a506-52afec82cc23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:23.531618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25cc1d-7ddf-49a1-a506-52afec82cc23 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:23.531899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25cc1d-7ddf-49a1-a506-52afec82cc23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:23.532167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25cc1d-7ddf-49a1-a506-52afec82cc23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:23.541226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25cc1d-7ddf-49a1-a506-52afec82cc23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:23.542722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25cc1d-7ddf-49a1-a506-52afec82cc23 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:23.552389 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1266/2561] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:23.823189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd43aa-e96e-49d3-86a4-70e2b739f09e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:23.824749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7bcd43aa-e96e-49d3-86a4-70e2b739f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:23.839418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd43aa-e96e-49d3-86a4-70e2b739f09e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:23.844349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:23.844984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:23.845268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:23.845563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] PATH_INFO: `/v3/projects/bc85b9a668974ee0a5c84af05486e8f0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:23.846742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc85b9a668974ee0a5c84af05486e8f0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:23.848722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:23.896001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc85b9a668974ee0a5c84af05486e8f0', 'request_id': 'req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.903363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc85b9a668974ee0a5c84af05486e8f0', 'request_id': 'req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.909730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc85b9a668974ee0a5c84af05486e8f0', 'request_id': 'req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.920389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] The token cache is being invalidate because project bc85b9a668974ee0a5c84af05486e8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:23.920696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.921322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:23.922842 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1296/2562] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:23 2022] DELETE /identity/v3/projects/bc85b9a668974ee0a5c84af05486e8f0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:23.927300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25cc1d-7ddf-49a1-a506-52afec82cc23 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:23.936841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25cc1d-7ddf-49a1-a506-52afec82cc23 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:23.993863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:23.994508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:23.994783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:23.995068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] PATH_INFO: `/v3/users/85121f63338e438e8d061567dd5d1789` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:23.996177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=85121f63338e438e8d061567dd5d1789)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:24.002368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:24.053010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85121f63338e438e8d061567dd5d1789'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.059877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85121f63338e438e8d061567dd5d1789'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.060517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85121f63338e438e8d061567dd5d1789'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.061111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85121f63338e438e8d061567dd5d1789'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.068628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e0ff2892-50b3-4a26-97bc-3401b096ad5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85121f63338e438e8d061567dd5d1789'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.085551 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1267/2563] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:23 2022] DELETE /identity/v3/users/85121f63338e438e8d061567dd5d1789 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:24.357296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:24.358741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:24.372490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f957f-0d2e-4244-a768-8aa6b1329e2e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:24.395818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:24.434538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:24.435165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.435441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.435708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] PATH_INFO: `/v3/projects/4a8f63f0a5b3400797291f7be59894fe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.436803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8f63f0a5b3400797291f7be59894fe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:24.438743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:24.484711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8f63f0a5b3400797291f7be59894fe', 'request_id': 'req-49f3090a-74a7-4812-bcdd-568f3b3987ed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.492052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8f63f0a5b3400797291f7be59894fe', 'request_id': 'req-49f3090a-74a7-4812-bcdd-568f3b3987ed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.498311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8f63f0a5b3400797291f7be59894fe', 'request_id': 'req-49f3090a-74a7-4812-bcdd-568f3b3987ed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.508841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] The token cache is being invalidate because project 4a8f63f0a5b3400797291f7be59894fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:24.509125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.509776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-49f3090a-74a7-4812-bcdd-568f3b3987ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:24.511215 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1297/2564] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:24 2022] DELETE /identity/v3/projects/4a8f63f0a5b3400797291f7be59894fe => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:24.516344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf018-d135-416c-b7aa-dd36aecc11fd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.516859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf018-d135-416c-b7aa-dd36aecc11fd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.516999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf018-d135-416c-b7aa-dd36aecc11fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.537170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0c1cf018-d135-416c-b7aa-dd36aecc11fd None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:24.538310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0c1cf018-d135-416c-b7aa-dd36aecc11fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:24.580634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:24.603821 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1268/2565] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:24.612662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9b9d-ba4a-44e9-918b-4a6e6087079e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.613351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9b9d-ba4a-44e9-918b-4a6e6087079e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.613694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9b9d-ba4a-44e9-918b-4a6e6087079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.634312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bcda9b9d-ba4a-44e9-918b-4a6e6087079e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:24.635562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bcda9b9d-ba4a-44e9-918b-4a6e6087079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:24.643829 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1298/2566] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:43:24.648651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cf018-d135-416c-b7aa-dd36aecc11fd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:24.658354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cf018-d135-416c-b7aa-dd36aecc11fd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:24.663601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:24.664197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.664476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.664747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.665393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:43:24.665726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:24.667308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:24.678851 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1269/2567] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:43:24.683045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9b9d-ba4a-44e9-918b-4a6e6087079e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:24.692707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9b9d-ba4a-44e9-918b-4a6e6087079e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:24.697427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6c4f-1ca3-4a5f-bb70-afe4be471096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:24.698055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6c4f-1ca3-4a5f-bb70-afe4be471096 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.698337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6c4f-1ca3-4a5f-bb70-afe4be471096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.698607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6c4f-1ca3-4a5f-bb70-afe4be471096 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.699240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6c4f-1ca3-4a5f-bb70-afe4be471096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:24.701135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6c4f-1ca3-4a5f-bb70-afe4be471096 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:24.729009 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1299/2568] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:24 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:43:24.733213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:24.743137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8493-7225-4d91-ac50-655a35133643 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:24.798384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270f143-c425-410c-8840-266361312cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:24.798889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270f143-c425-410c-8840-266361312cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.799171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270f143-c425-410c-8840-266361312cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.799446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270f143-c425-410c-8840-266361312cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.800068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270f143-c425-410c-8840-266361312cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:24.802269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270f143-c425-410c-8840-266361312cf2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:24.823182 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1270/2569] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:24 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:24.827417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6c4f-1ca3-4a5f-bb70-afe4be471096 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:24.838604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6c4f-1ca3-4a5f-bb70-afe4be471096 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:24.843137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dc9cd-da83-4c52-9944-99f1ef6674fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:24.843723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dc9cd-da83-4c52-9944-99f1ef6674fb admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.844003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dc9cd-da83-4c52-9944-99f1ef6674fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.844273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dc9cd-da83-4c52-9944-99f1ef6674fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.844906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573dc9cd-da83-4c52-9944-99f1ef6674fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:24.846546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573dc9cd-da83-4c52-9944-99f1ef6674fb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:24.856441 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1300/2570] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:24.860253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270f143-c425-410c-8840-266361312cf2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:24.870151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270f143-c425-410c-8840-266361312cf2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:24.874820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f441f-e4ab-4930-97d9-bee501a6edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:24.875410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f441f-e4ab-4930-97d9-bee501a6edcc admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.875687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f441f-e4ab-4930-97d9-bee501a6edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.875955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f441f-e4ab-4930-97d9-bee501a6edcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.876644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f441f-e4ab-4930-97d9-bee501a6edcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:24.878307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f441f-e4ab-4930-97d9-bee501a6edcc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:24.889162 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1271/2571] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:24.893568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dc9cd-da83-4c52-9944-99f1ef6674fb admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:24.905043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dc9cd-da83-4c52-9944-99f1ef6674fb admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:24.909619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02be9-7f8a-4951-af9a-3d151910168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:24.910287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02be9-7f8a-4951-af9a-3d151910168a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.910559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02be9-7f8a-4951-af9a-3d151910168a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.910828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02be9-7f8a-4951-af9a-3d151910168a admin admin] PATH_INFO: `/v3/projects/02dc86da263947619789872b30627b9e/users/d1693d631a524385a6ef5a9da1783901/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.924727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac02be9-7f8a-4951-af9a-3d151910168a admin admin] RBAC: Authorizing `identity:create_grant(project_id=02dc86da263947619789872b30627b9e, user_id=d1693d631a524385a6ef5a9da1783901, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:24.927411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac02be9-7f8a-4951-af9a-3d151910168a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:24.940556 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1301/2572] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:24 2022] PUT /identity/v3/projects/02dc86da263947619789872b30627b9e/users/d1693d631a524385a6ef5a9da1783901/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:24.951464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd662ea-d034-4d33-8df1-0f282aa8dc81 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:24.951752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd662ea-d034-4d33-8df1-0f282aa8dc81 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:24.952029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd662ea-d034-4d33-8df1-0f282aa8dc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:24.979732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-efd662ea-d034-4d33-8df1-0f282aa8dc81 None None] MFA Rules not processed for user `d1693d631a524385a6ef5a9da1783901`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:24.981169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-efd662ea-d034-4d33-8df1-0f282aa8dc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:25.044180 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1272/2573] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:25.048978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02be9-7f8a-4951-af9a-3d151910168a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.100586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:25.114332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02be9-7f8a-4951-af9a-3d151910168a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.119232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de7a3-47d9-4233-ba8c-81dde2460381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.119885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de7a3-47d9-4233-ba8c-81dde2460381 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.120289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de7a3-47d9-4233-ba8c-81dde2460381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.120514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de7a3-47d9-4233-ba8c-81dde2460381 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.121315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4de7a3-47d9-4233-ba8c-81dde2460381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.123407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4de7a3-47d9-4233-ba8c-81dde2460381 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.150709 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1302/2574] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:25 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:25.154832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd662ea-d034-4d33-8df1-0f282aa8dc81 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.165359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd662ea-d034-4d33-8df1-0f282aa8dc81 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.198672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:25.222377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390f13d-6c29-44e5-90b4-6d5253a659ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.222974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390f13d-6c29-44e5-90b4-6d5253a659ca admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.223249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390f13d-6c29-44e5-90b4-6d5253a659ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.223525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390f13d-6c29-44e5-90b4-6d5253a659ca admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.224142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390f13d-6c29-44e5-90b4-6d5253a659ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.226079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390f13d-6c29-44e5-90b4-6d5253a659ca admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.246666 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1273/2575] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:25 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:25.250561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de7a3-47d9-4233-ba8c-81dde2460381 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.261139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de7a3-47d9-4233-ba8c-81dde2460381 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.265623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2f4ac-b3fb-4bcd-9a4f-2be83eadb592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.266488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f4ac-b3fb-4bcd-9a4f-2be83eadb592 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.266764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f4ac-b3fb-4bcd-9a4f-2be83eadb592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.267043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f4ac-b3fb-4bcd-9a4f-2be83eadb592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.267665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2f4ac-b3fb-4bcd-9a4f-2be83eadb592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.269370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2f4ac-b3fb-4bcd-9a4f-2be83eadb592 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.279033 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1303/2576] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:43:25.282786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390f13d-6c29-44e5-90b4-6d5253a659ca admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.292761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390f13d-6c29-44e5-90b4-6d5253a659ca admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.297390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7350dbb-aed5-4cea-9c4b-5eedd7694047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.298016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350dbb-aed5-4cea-9c4b-5eedd7694047 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.298289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350dbb-aed5-4cea-9c4b-5eedd7694047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.298557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7350dbb-aed5-4cea-9c4b-5eedd7694047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.299241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7350dbb-aed5-4cea-9c4b-5eedd7694047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.301312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7350dbb-aed5-4cea-9c4b-5eedd7694047 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.311037 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1274/2577] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:25.315013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2f4ac-b3fb-4bcd-9a4f-2be83eadb592 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.325828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2f4ac-b3fb-4bcd-9a4f-2be83eadb592 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.331024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73715994-043c-46fe-9d31-8ba78a0bbf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.331678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73715994-043c-46fe-9d31-8ba78a0bbf2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.332069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73715994-043c-46fe-9d31-8ba78a0bbf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.332301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73715994-043c-46fe-9d31-8ba78a0bbf2b admin admin] PATH_INFO: `/v3/projects/dd0349566d8040789dfab0304b96bfa4/users/e2ae0a54bb58436fbeae37469e885df7/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.347784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73715994-043c-46fe-9d31-8ba78a0bbf2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0349566d8040789dfab0304b96bfa4, user_id=e2ae0a54bb58436fbeae37469e885df7, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.350490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73715994-043c-46fe-9d31-8ba78a0bbf2b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.363706 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1304/2578] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:25 2022] PUT /identity/v3/projects/dd0349566d8040789dfab0304b96bfa4/users/e2ae0a54bb58436fbeae37469e885df7/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:25.373970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2de170-69ae-4c79-9eca-d2a4706fa6f9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.374254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2de170-69ae-4c79-9eca-d2a4706fa6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.374524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2de170-69ae-4c79-9eca-d2a4706fa6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.402997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cd2de170-69ae-4c79-9eca-d2a4706fa6f9 None None] MFA Rules not processed for user `e2ae0a54bb58436fbeae37469e885df7`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:25.404234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cd2de170-69ae-4c79-9eca-d2a4706fa6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:25.469670 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1275/2579] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:43:25.474768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73715994-043c-46fe-9d31-8ba78a0bbf2b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.535724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73715994-043c-46fe-9d31-8ba78a0bbf2b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.540650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68179452-1fe9-4493-92a3-85a48319dd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.541246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179452-1fe9-4493-92a3-85a48319dd80 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.541527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179452-1fe9-4493-92a3-85a48319dd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.541839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179452-1fe9-4493-92a3-85a48319dd80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.542485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68179452-1fe9-4493-92a3-85a48319dd80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.544473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68179452-1fe9-4493-92a3-85a48319dd80 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.570459 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1305/2580] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:25 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:25.574411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2de170-69ae-4c79-9eca-d2a4706fa6f9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.585140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2de170-69ae-4c79-9eca-d2a4706fa6f9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.640825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256b069-a39d-4a44-b944-8f4cf95ef120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.641702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b069-a39d-4a44-b944-8f4cf95ef120 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.641986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b069-a39d-4a44-b944-8f4cf95ef120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.642261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256b069-a39d-4a44-b944-8f4cf95ef120 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.642894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256b069-a39d-4a44-b944-8f4cf95ef120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.645166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256b069-a39d-4a44-b944-8f4cf95ef120 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.667211 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1276/2581] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:25 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:25.671212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68179452-1fe9-4493-92a3-85a48319dd80 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.681919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68179452-1fe9-4493-92a3-85a48319dd80 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.687196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629f073-b473-43db-bb3c-85e851c03519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.687981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629f073-b473-43db-bb3c-85e851c03519 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.688456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629f073-b473-43db-bb3c-85e851c03519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.688815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629f073-b473-43db-bb3c-85e851c03519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.689571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629f073-b473-43db-bb3c-85e851c03519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.691357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629f073-b473-43db-bb3c-85e851c03519 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.702365 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1306/2582] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:25.706755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256b069-a39d-4a44-b944-8f4cf95ef120 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.717551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256b069-a39d-4a44-b944-8f4cf95ef120 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.722144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d69709-e829-43e7-970d-b4c677979ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.723389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d69709-e829-43e7-970d-b4c677979ede admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.723665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d69709-e829-43e7-970d-b4c677979ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.723933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d69709-e829-43e7-970d-b4c677979ede admin admin] PATH_INFO: `/v3/projects/14096e54f9ee4e95a65f0de56d7c66bd/users/20095f44cbf4406d8c44a1433e7f11cf/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.739008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d69709-e829-43e7-970d-b4c677979ede admin admin] RBAC: Authorizing `identity:create_grant(project_id=14096e54f9ee4e95a65f0de56d7c66bd, user_id=20095f44cbf4406d8c44a1433e7f11cf, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.741685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d69709-e829-43e7-970d-b4c677979ede admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.754048 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1277/2583] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:25 2022] PUT /identity/v3/projects/14096e54f9ee4e95a65f0de56d7c66bd/users/20095f44cbf4406d8c44a1433e7f11cf/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:43:25.758037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629f073-b473-43db-bb3c-85e851c03519 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.768106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629f073-b473-43db-bb3c-85e851c03519 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.824340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a68cd6-fd39-4ce4-b25d-94d4695ca2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.824862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a68cd6-fd39-4ce4-b25d-94d4695ca2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.825141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a68cd6-fd39-4ce4-b25d-94d4695ca2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.825409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a68cd6-fd39-4ce4-b25d-94d4695ca2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.826066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a68cd6-fd39-4ce4-b25d-94d4695ca2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.827676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a68cd6-fd39-4ce4-b25d-94d4695ca2f2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.837550 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1307/2584] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:25.841694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d69709-e829-43e7-970d-b4c677979ede admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.852214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d69709-e829-43e7-970d-b4c677979ede admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:25.856644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:25.857280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.857561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.857866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] PATH_INFO: `/v3/projects/14096e54f9ee4e95a65f0de56d7c66bd/users/20095f44cbf4406d8c44a1433e7f11cf/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.859747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14096e54f9ee4e95a65f0de56d7c66bd, user_id=20095f44cbf4406d8c44a1433e7f11cf, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:25.862396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:25.868326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:25.874506 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1278/2585] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:25 2022] PUT /identity/v3/projects/14096e54f9ee4e95a65f0de56d7c66bd/users/20095f44cbf4406d8c44a1433e7f11cf/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:25.885772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216d158-04a1-4057-a4dc-90b0493cf635 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:25.886057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216d158-04a1-4057-a4dc-90b0493cf635 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:25.886330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216d158-04a1-4057-a4dc-90b0493cf635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:25.907295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:25.916027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e216d158-04a1-4057-a4dc-90b0493cf635 None None] MFA Rules not processed for user `20095f44cbf4406d8c44a1433e7f11cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:25.918168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e216d158-04a1-4057-a4dc-90b0493cf635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:25.986582 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1308/2586] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:25.997934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:25.999680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:26.063453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eb9b8-4fdb-4509-8b57-9527827a5e82 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:26.067910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d31127-b408-4e34-909d-21b3330fb361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:26.068508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31127-b408-4e34-909d-21b3330fb361 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:26.068783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31127-b408-4e34-909d-21b3330fb361 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:26.069056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31127-b408-4e34-909d-21b3330fb361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:26.079834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d31127-b408-4e34-909d-21b3330fb361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:26.081187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d31127-b408-4e34-909d-21b3330fb361 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:26.091196 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1279/2587] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:26.133608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216d158-04a1-4057-a4dc-90b0493cf635 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:26.135009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e216d158-04a1-4057-a4dc-90b0493cf635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:26.148338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216d158-04a1-4057-a4dc-90b0493cf635 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:26.151894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106d3da-4707-40e2-a083-fdc6969415d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:26.152486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106d3da-4707-40e2-a083-fdc6969415d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:26.152764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106d3da-4707-40e2-a083-fdc6969415d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:26.153033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106d3da-4707-40e2-a083-fdc6969415d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:26.210613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106d3da-4707-40e2-a083-fdc6969415d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:26.212071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106d3da-4707-40e2-a083-fdc6969415d2 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:26.221120 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1309/2588] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:29.102630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d31127-b408-4e34-909d-21b3330fb361 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:29.164877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d31127-b408-4e34-909d-21b3330fb361 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:29.169584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:29.170247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:29.170528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:29.170798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] PATH_INFO: `/v3/users/d1693d631a524385a6ef5a9da1783901` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:29.171935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1693d631a524385a6ef5a9da1783901)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:29.173921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:29.219855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1693d631a524385a6ef5a9da1783901'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.227119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1693d631a524385a6ef5a9da1783901'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.227768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1693d631a524385a6ef5a9da1783901'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.228364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1693d631a524385a6ef5a9da1783901'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.235794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1693d631a524385a6ef5a9da1783901'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.253989 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1280/2589] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:29 2022] DELETE /identity/v3/users/d1693d631a524385a6ef5a9da1783901 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:29.258789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3954b-aeb7-48ac-ad84-3b6fae1edb1a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:29.259260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3954b-aeb7-48ac-ad84-3b6fae1edb1a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:29.259456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3954b-aeb7-48ac-ad84-3b6fae1edb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:29.279612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c1e3954b-aeb7-48ac-ad84-3b6fae1edb1a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:29.280770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c1e3954b-aeb7-48ac-ad84-3b6fae1edb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:29.352874 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1310/2590] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:29.362520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:29.364630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:29.386071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:29.424172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc65721-2686-4ead-baec-4bbdff058844 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:29.428647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1433a-6006-4dc4-bdcb-aad161f9ccc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:29.429247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1433a-6006-4dc4-bdcb-aad161f9ccc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:29.429522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1433a-6006-4dc4-bdcb-aad161f9ccc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:29.429822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1433a-6006-4dc4-bdcb-aad161f9ccc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:29.438486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1433a-6006-4dc4-bdcb-aad161f9ccc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:29.439898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1433a-6006-4dc4-bdcb-aad161f9ccc5 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:29.449584 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1281/2591] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:29.727975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3954b-aeb7-48ac-ad84-3b6fae1edb1a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:29.729421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c1e3954b-aeb7-48ac-ad84-3b6fae1edb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:29.743583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3954b-aeb7-48ac-ad84-3b6fae1edb1a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:29.748533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:29.749163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:29.749442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:29.749743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] PATH_INFO: `/v3/projects/02dc86da263947619789872b30627b9e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:29.750843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02dc86da263947619789872b30627b9e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:29.752824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:29.788955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:29.805328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02dc86da263947619789872b30627b9e', 'request_id': 'req-3a5452ab-5aed-489a-b354-9b6def3d9849'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.813113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02dc86da263947619789872b30627b9e', 'request_id': 'req-3a5452ab-5aed-489a-b354-9b6def3d9849'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.820707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02dc86da263947619789872b30627b9e', 'request_id': 'req-3a5452ab-5aed-489a-b354-9b6def3d9849'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.832688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] The token cache is being invalidate because project 02dc86da263947619789872b30627b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:29.833088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.833608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.835119 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1311/2592] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:29 2022] DELETE /identity/v3/projects/02dc86da263947619789872b30627b9e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:29.839307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1433a-6006-4dc4-bdcb-aad161f9ccc5 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:29.850092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1433a-6006-4dc4-bdcb-aad161f9ccc5 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:29.911527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:29.912159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:29.912433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:29.912700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] PATH_INFO: `/v3/users/e2ae0a54bb58436fbeae37469e885df7` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:29.913842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ae0a54bb58436fbeae37469e885df7)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:29.916120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:29.965551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ae0a54bb58436fbeae37469e885df7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.974045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ae0a54bb58436fbeae37469e885df7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.974945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ae0a54bb58436fbeae37469e885df7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.975429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ae0a54bb58436fbeae37469e885df7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:29.983375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ae0a54bb58436fbeae37469e885df7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.002026 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1282/2593] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:29 2022] DELETE /identity/v3/users/e2ae0a54bb58436fbeae37469e885df7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:30.296235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:30.297365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:30.315672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5452ab-5aed-489a-b354-9b6def3d9849 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:30.377221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:30.377883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:30.378969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:30.379253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] PATH_INFO: `/v3/projects/dd0349566d8040789dfab0304b96bfa4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:30.380364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0349566d8040789dfab0304b96bfa4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:30.382301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:30.432311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0349566d8040789dfab0304b96bfa4', 'request_id': 'req-99da5dec-2ec7-4307-b315-a2188cfaa91e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.440006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0349566d8040789dfab0304b96bfa4', 'request_id': 'req-99da5dec-2ec7-4307-b315-a2188cfaa91e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.447213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0349566d8040789dfab0304b96bfa4', 'request_id': 'req-99da5dec-2ec7-4307-b315-a2188cfaa91e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.457570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] The token cache is being invalidate because project dd0349566d8040789dfab0304b96bfa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:30.457904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.458546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.460016 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1312/2594] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:30 2022] DELETE /identity/v3/projects/dd0349566d8040789dfab0304b96bfa4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:30.464161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:30.465688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:30.479752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c57b12-bf07-4dad-9e1e-a78cd321bb9e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:30.522781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:30.540866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:30.541502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:30.541848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:30.542123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] PATH_INFO: `/v3/users/20095f44cbf4406d8c44a1433e7f11cf` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:30.543200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20095f44cbf4406d8c44a1433e7f11cf)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:30.545096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:30.591621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20095f44cbf4406d8c44a1433e7f11cf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.600469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20095f44cbf4406d8c44a1433e7f11cf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.601908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20095f44cbf4406d8c44a1433e7f11cf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.601908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20095f44cbf4406d8c44a1433e7f11cf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.609887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-255d7e0c-99f3-4c68-b6e9-15512c460f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20095f44cbf4406d8c44a1433e7f11cf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:30.627554 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1283/2595] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:30 2022] DELETE /identity/v3/users/20095f44cbf4406d8c44a1433e7f11cf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:30.895470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:30.896897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:30.910522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da5dec-2ec7-4307-b315-a2188cfaa91e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:30.960590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:30.977141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:30.977823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:30.978104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:30.978375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] PATH_INFO: `/v3/projects/14096e54f9ee4e95a65f0de56d7c66bd` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:30.979607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] RBAC: Authorizing `identity:delete_project(project_id=14096e54f9ee4e95a65f0de56d7c66bd)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:30.981515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.027439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14096e54f9ee4e95a65f0de56d7c66bd', 'request_id': 'req-6800fedf-60f7-4e2e-bf82-febbcc9219be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:31.035107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14096e54f9ee4e95a65f0de56d7c66bd', 'request_id': 'req-6800fedf-60f7-4e2e-bf82-febbcc9219be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:31.041430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14096e54f9ee4e95a65f0de56d7c66bd', 'request_id': 'req-6800fedf-60f7-4e2e-bf82-febbcc9219be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:31.053067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] The token cache is being invalidate because project 14096e54f9ee4e95a65f0de56d7c66bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:31.053353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:31.054003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6800fedf-60f7-4e2e-bf82-febbcc9219be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:31.055465 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1313/2596] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:30 2022] DELETE /identity/v3/projects/14096e54f9ee4e95a65f0de56d7c66bd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:31.061431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d344032-6f99-41f0-a26c-f397c31d3b80 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.061742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d344032-6f99-41f0-a26c-f397c31d3b80 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.062019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d344032-6f99-41f0-a26c-f397c31d3b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.081509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6d344032-6f99-41f0-a26c-f397c31d3b80 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:31.082758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6d344032-6f99-41f0-a26c-f397c31d3b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:31.142401 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1284/2597] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:31.150906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73622cc-9117-4d4b-80fc-4cbb0bee3d06 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.151272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73622cc-9117-4d4b-80fc-4cbb0bee3d06 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.151663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73622cc-9117-4d4b-80fc-4cbb0bee3d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.172570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a73622cc-9117-4d4b-80fc-4cbb0bee3d06 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:31.173818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a73622cc-9117-4d4b-80fc-4cbb0bee3d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:31.181950 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1314/2598] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:31.186089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d344032-6f99-41f0-a26c-f397c31d3b80 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.196015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d344032-6f99-41f0-a26c-f397c31d3b80 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.200805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.201404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.201706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.201976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.202672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:43:31.202972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.204557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.216663 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1285/2599] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:43:31.221494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73622cc-9117-4d4b-80fc-4cbb0bee3d06 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.231473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73622cc-9117-4d4b-80fc-4cbb0bee3d06 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.236084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b34fe-8d0c-46e1-9d6c-30db4b380477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.236683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b34fe-8d0c-46e1-9d6c-30db4b380477 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.236961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b34fe-8d0c-46e1-9d6c-30db4b380477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.237230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b34fe-8d0c-46e1-9d6c-30db4b380477 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.238037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b34fe-8d0c-46e1-9d6c-30db4b380477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.239933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b34fe-8d0c-46e1-9d6c-30db4b380477 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.273093 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1315/2600] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:31 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:31.274426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.284518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28808e77-df48-4988-bd83-eb804bc7cf2a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.328610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:31.342365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc175f7-94f8-4ba1-a53b-431e9452e0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.342970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc175f7-94f8-4ba1-a53b-431e9452e0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.343250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc175f7-94f8-4ba1-a53b-431e9452e0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.343527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc175f7-94f8-4ba1-a53b-431e9452e0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.344151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc175f7-94f8-4ba1-a53b-431e9452e0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.346117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc175f7-94f8-4ba1-a53b-431e9452e0c6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.367125 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1286/2601] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:31 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:31.371323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b34fe-8d0c-46e1-9d6c-30db4b380477 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.382141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b34fe-8d0c-46e1-9d6c-30db4b380477 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.386734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a5738-54a9-47d6-bf46-8c98ec9ee8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.387330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a5738-54a9-47d6-bf46-8c98ec9ee8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.387610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a5738-54a9-47d6-bf46-8c98ec9ee8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.387879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a5738-54a9-47d6-bf46-8c98ec9ee8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.388519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a5738-54a9-47d6-bf46-8c98ec9ee8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.394471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a5738-54a9-47d6-bf46-8c98ec9ee8f7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.401452 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1316/2602] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:31.405562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc175f7-94f8-4ba1-a53b-431e9452e0c6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.416022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc175f7-94f8-4ba1-a53b-431e9452e0c6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.420529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6352dd-bee5-4ad3-9881-4e731ce195ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.421116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6352dd-bee5-4ad3-9881-4e731ce195ae admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.421392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6352dd-bee5-4ad3-9881-4e731ce195ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.421687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6352dd-bee5-4ad3-9881-4e731ce195ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.422321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6352dd-bee5-4ad3-9881-4e731ce195ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.423897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6352dd-bee5-4ad3-9881-4e731ce195ae admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.434612 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1287/2603] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:31.438669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a5738-54a9-47d6-bf46-8c98ec9ee8f7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.448536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a5738-54a9-47d6-bf46-8c98ec9ee8f7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.453187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e417f76-222c-4872-b9d2-02d90e329d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.453856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417f76-222c-4872-b9d2-02d90e329d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.454135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417f76-222c-4872-b9d2-02d90e329d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.454403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e417f76-222c-4872-b9d2-02d90e329d82 admin admin] PATH_INFO: `/v3/projects/4852f03e33964ba28f0e83d091abba9a/users/8375c05eee4542f8be380030aae548f0/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.468926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e417f76-222c-4872-b9d2-02d90e329d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4852f03e33964ba28f0e83d091abba9a, user_id=8375c05eee4542f8be380030aae548f0, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.471599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e417f76-222c-4872-b9d2-02d90e329d82 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.484955 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1317/2604] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:31 2022] PUT /identity/v3/projects/4852f03e33964ba28f0e83d091abba9a/users/8375c05eee4542f8be380030aae548f0/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:43:31.497224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650407-1aee-48a6-9a93-b7973d43bce6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.497739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650407-1aee-48a6-9a93-b7973d43bce6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.497878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650407-1aee-48a6-9a93-b7973d43bce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.526218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-27650407-1aee-48a6-9a93-b7973d43bce6 None None] MFA Rules not processed for user `8375c05eee4542f8be380030aae548f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:31.527405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-27650407-1aee-48a6-9a93-b7973d43bce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:31.594871 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1288/2605] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:31.599705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e417f76-222c-4872-b9d2-02d90e329d82 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.663690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e417f76-222c-4872-b9d2-02d90e329d82 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.668313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8faaa-7710-4f16-ab77-6af413990f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.668911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8faaa-7710-4f16-ab77-6af413990f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.669185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8faaa-7710-4f16-ab77-6af413990f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.669449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8faaa-7710-4f16-ab77-6af413990f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.670153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8faaa-7710-4f16-ab77-6af413990f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.672181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8faaa-7710-4f16-ab77-6af413990f0b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.696024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:31.705728 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1318/2606] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:31 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:31.709281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27650407-1aee-48a6-9a93-b7973d43bce6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.719877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27650407-1aee-48a6-9a93-b7973d43bce6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.776358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c5c16-d903-49f6-98d9-7e5546535355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.776967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c5c16-d903-49f6-98d9-7e5546535355 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.777243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c5c16-d903-49f6-98d9-7e5546535355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.777511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c5c16-d903-49f6-98d9-7e5546535355 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.778190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c5c16-d903-49f6-98d9-7e5546535355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.780080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c5c16-d903-49f6-98d9-7e5546535355 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.799683 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1289/2607] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:31 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:31.803669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8faaa-7710-4f16-ab77-6af413990f0b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.813799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8faaa-7710-4f16-ab77-6af413990f0b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.818907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8ac91-d924-4e04-820d-b9431db0ef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.819491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ac91-d924-4e04-820d-b9431db0ef13 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.819768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ac91-d924-4e04-820d-b9431db0ef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.820036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8ac91-d924-4e04-820d-b9431db0ef13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.820695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8ac91-d924-4e04-820d-b9431db0ef13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.822327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8ac91-d924-4e04-820d-b9431db0ef13 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.834488 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1319/2608] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:31.837867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c5c16-d903-49f6-98d9-7e5546535355 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.847870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c5c16-d903-49f6-98d9-7e5546535355 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.852528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187aae9-0e74-4e29-9691-697aac864278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.853176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187aae9-0e74-4e29-9691-697aac864278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.853458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187aae9-0e74-4e29-9691-697aac864278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.853767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187aae9-0e74-4e29-9691-697aac864278 admin admin] PATH_INFO: `/v3/projects/ff1159387a934e67957311fe699c9243/users/f08964318c57495382134961dd3081f9/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.868727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187aae9-0e74-4e29-9691-697aac864278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1159387a934e67957311fe699c9243, user_id=f08964318c57495382134961dd3081f9, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.871423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187aae9-0e74-4e29-9691-697aac864278 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.884597 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1290/2609] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:31 2022] PUT /identity/v3/projects/ff1159387a934e67957311fe699c9243/users/f08964318c57495382134961dd3081f9/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:31.888539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8ac91-d924-4e04-820d-b9431db0ef13 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.899904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8ac91-d924-4e04-820d-b9431db0ef13 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.956231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f4a1c-2975-4f47-b52c-7cce89bfc761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.960248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4a1c-2975-4f47-b52c-7cce89bfc761 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.960248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4a1c-2975-4f47-b52c-7cce89bfc761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.960248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4a1c-2975-4f47-b52c-7cce89bfc761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.960248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f4a1c-2975-4f47-b52c-7cce89bfc761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:31.960248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f4a1c-2975-4f47-b52c-7cce89bfc761 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:31.971496 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1320/2610] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:31.975773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187aae9-0e74-4e29-9691-697aac864278 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:31.986522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187aae9-0e74-4e29-9691-697aac864278 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:31.992351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:31.993012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:31.993366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:31.993687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] PATH_INFO: `/v3/projects/ff1159387a934e67957311fe699c9243/users/f08964318c57495382134961dd3081f9/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:31.997386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1159387a934e67957311fe699c9243, user_id=f08964318c57495382134961dd3081f9, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:32.000406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:32.011604 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1291/2611] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:31 2022] PUT /identity/v3/projects/ff1159387a934e67957311fe699c9243/users/f08964318c57495382134961dd3081f9/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:32.076809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effc42f-a915-40ce-a446-49c71b5adaa4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:32.077193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effc42f-a915-40ce-a446-49c71b5adaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:32.077566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effc42f-a915-40ce-a446-49c71b5adaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:32.105851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2effc42f-a915-40ce-a446-49c71b5adaa4 None None] MFA Rules not processed for user `f08964318c57495382134961dd3081f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:32.107556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2effc42f-a915-40ce-a446-49c71b5adaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:32.182043 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1321/2612] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:32.194471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:32.196319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:32.213446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:32.269467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d79fcd-fc42-4ede-8ae3-92e4de8ab1ae admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:32.273209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5c9a8-ed2e-4f3f-a70d-8fa9a694ec17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:32.273927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c9a8-ed2e-4f3f-a70d-8fa9a694ec17 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:32.274207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c9a8-ed2e-4f3f-a70d-8fa9a694ec17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:32.274481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c9a8-ed2e-4f3f-a70d-8fa9a694ec17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:32.285494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5c9a8-ed2e-4f3f-a70d-8fa9a694ec17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:32.287092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5c9a8-ed2e-4f3f-a70d-8fa9a694ec17 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:32.301433 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1292/2613] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:32.393684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effc42f-a915-40ce-a446-49c71b5adaa4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:32.395351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2effc42f-a915-40ce-a446-49c71b5adaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:32.415678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effc42f-a915-40ce-a446-49c71b5adaa4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:32.419328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada66df-e360-4c3e-86a4-85f09c44142c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:32.419936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada66df-e360-4c3e-86a4-85f09c44142c service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:32.420221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada66df-e360-4c3e-86a4-85f09c44142c service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:32.420496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada66df-e360-4c3e-86a4-85f09c44142c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:32.479644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada66df-e360-4c3e-86a4-85f09c44142c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:32.481122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada66df-e360-4c3e-86a4-85f09c44142c service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:32.490634 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1322/2614] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:37.293810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5c9a8-ed2e-4f3f-a70d-8fa9a694ec17 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:37.354052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5c9a8-ed2e-4f3f-a70d-8fa9a694ec17 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:37.358584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:37.359208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:37.359488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:37.359760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] PATH_INFO: `/v3/users/8375c05eee4542f8be380030aae548f0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:37.360864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8375c05eee4542f8be380030aae548f0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:37.362848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:37.410828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8375c05eee4542f8be380030aae548f0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:37.418259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8375c05eee4542f8be380030aae548f0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:37.418916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8375c05eee4542f8be380030aae548f0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:37.419514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8375c05eee4542f8be380030aae548f0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:37.426915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8375c05eee4542f8be380030aae548f0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:37.444741 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1293/2615] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:37 2022] DELETE /identity/v3/users/8375c05eee4542f8be380030aae548f0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:37.449320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429083fe-f044-4202-a6aa-219f64d1b2ea None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:37.449852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429083fe-f044-4202-a6aa-219f64d1b2ea None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:37.449973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429083fe-f044-4202-a6aa-219f64d1b2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:37.469433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-429083fe-f044-4202-a6aa-219f64d1b2ea None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:37.470594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-429083fe-f044-4202-a6aa-219f64d1b2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:37.474721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:37.534773 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1323/2616] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:43:37.544712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:37.546205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:37.585583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:37.614321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77db70-377b-49c0-bf47-7b682d4acaad admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:37.618561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd619a-a1ee-4ce0-9406-0fd17952e7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:37.619174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd619a-a1ee-4ce0-9406-0fd17952e7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:37.620405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd619a-a1ee-4ce0-9406-0fd17952e7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:37.620405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd619a-a1ee-4ce0-9406-0fd17952e7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:37.629377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd619a-a1ee-4ce0-9406-0fd17952e7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:37.631699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd619a-a1ee-4ce0-9406-0fd17952e7c1 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:37.642774 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1294/2617] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:43:38.040625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429083fe-f044-4202-a6aa-219f64d1b2ea None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:38.043734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-429083fe-f044-4202-a6aa-219f64d1b2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:38.060317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429083fe-f044-4202-a6aa-219f64d1b2ea None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:38.066838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:38.067646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:38.067996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:38.068857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] PATH_INFO: `/v3/projects/4852f03e33964ba28f0e83d091abba9a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:38.069885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4852f03e33964ba28f0e83d091abba9a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:38.072097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:38.121122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4852f03e33964ba28f0e83d091abba9a', 'request_id': 'req-02028043-e2b3-4acc-9f20-f3af332af8e0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.129041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4852f03e33964ba28f0e83d091abba9a', 'request_id': 'req-02028043-e2b3-4acc-9f20-f3af332af8e0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.135955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4852f03e33964ba28f0e83d091abba9a', 'request_id': 'req-02028043-e2b3-4acc-9f20-f3af332af8e0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.147931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] The token cache is being invalidate because project 4852f03e33964ba28f0e83d091abba9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:38.148222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.148872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.150507 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1324/2618] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:38 2022] DELETE /identity/v3/projects/4852f03e33964ba28f0e83d091abba9a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:38.154453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd619a-a1ee-4ce0-9406-0fd17952e7c1 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:38.164764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd619a-a1ee-4ce0-9406-0fd17952e7c1 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:38.222277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:38.222917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:38.223201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:38.223478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] PATH_INFO: `/v3/users/f08964318c57495382134961dd3081f9` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:38.224660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08964318c57495382134961dd3081f9)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:38.226659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:38.279723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08964318c57495382134961dd3081f9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.287518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08964318c57495382134961dd3081f9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.288300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08964318c57495382134961dd3081f9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.289017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08964318c57495382134961dd3081f9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.298627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dc942aff-b760-42c6-bc48-6a4e8574a852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08964318c57495382134961dd3081f9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.322976 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1295/2619] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:38 2022] DELETE /identity/v3/users/f08964318c57495382134961dd3081f9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:38.658884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:38.660544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:38.676565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02028043-e2b3-4acc-9f20-f3af332af8e0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:38.756215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:38.757655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:38.758508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:38.758783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] PATH_INFO: `/v3/projects/ff1159387a934e67957311fe699c9243` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:38.760025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1159387a934e67957311fe699c9243)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:38.762002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:38.802566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:38.814380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1159387a934e67957311fe699c9243', 'request_id': 'req-8c3f02c0-d805-419f-b748-a99ceb2a195d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.824375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1159387a934e67957311fe699c9243', 'request_id': 'req-8c3f02c0-d805-419f-b748-a99ceb2a195d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.831243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1159387a934e67957311fe699c9243', 'request_id': 'req-8c3f02c0-d805-419f-b748-a99ceb2a195d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.846366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] The token cache is being invalidate because project ff1159387a934e67957311fe699c9243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:38.846717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.847450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8c3f02c0-d805-419f-b748-a99ceb2a195d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:38.849176 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1325/2620] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:38 2022] DELETE /identity/v3/projects/ff1159387a934e67957311fe699c9243 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:38.854053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e465-943d-4f21-8f67-d154e86546f6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:38.854970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e465-943d-4f21-8f67-d154e86546f6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:38.855320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5e465-943d-4f21-8f67-d154e86546f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:38.875186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-dab5e465-943d-4f21-8f67-d154e86546f6 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:38.876490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dab5e465-943d-4f21-8f67-d154e86546f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:38.930314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:38.946545 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1296/2621] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:38.954349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc3c05-bf10-47aa-8064-fd443e218360 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:38.954630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc3c05-bf10-47aa-8064-fd443e218360 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:38.954900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc3c05-bf10-47aa-8064-fd443e218360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:38.978145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0ddc3c05-bf10-47aa-8064-fd443e218360 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:38.978795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0ddc3c05-bf10-47aa-8064-fd443e218360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:38.999124 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1326/2622] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:39.000422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5e465-943d-4f21-8f67-d154e86546f6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:39.012122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5e465-943d-4f21-8f67-d154e86546f6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:39.017819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:39.019699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:39.020335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:39.021252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:39.022494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:43:39.023504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:39.027054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:39.048468 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1297/2623] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:43:39.052990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc3c05-bf10-47aa-8064-fd443e218360 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:39.062695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc3c05-bf10-47aa-8064-fd443e218360 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:39.068347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f95d3-23c4-4526-8df5-4179ae61715d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:39.069449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f95d3-23c4-4526-8df5-4179ae61715d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:39.069763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f95d3-23c4-4526-8df5-4179ae61715d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:39.070037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f95d3-23c4-4526-8df5-4179ae61715d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:39.070692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f95d3-23c4-4526-8df5-4179ae61715d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:39.072642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f95d3-23c4-4526-8df5-4179ae61715d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:39.112709 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1327/2624] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:39 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:39.119508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:39.134615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26fe81-9640-4faf-ae91-1719bfe7d262 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:39.207333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99beac53-ee14-45a3-a3ae-9ed93ed1e404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:39.208813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99beac53-ee14-45a3-a3ae-9ed93ed1e404 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:39.209176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99beac53-ee14-45a3-a3ae-9ed93ed1e404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:39.209525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99beac53-ee14-45a3-a3ae-9ed93ed1e404 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:39.210224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99beac53-ee14-45a3-a3ae-9ed93ed1e404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:39.212264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99beac53-ee14-45a3-a3ae-9ed93ed1e404 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:39.234197 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1298/2625] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:39 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:39.238357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f95d3-23c4-4526-8df5-4179ae61715d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:39.249326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f95d3-23c4-4526-8df5-4179ae61715d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:39.255394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc220400-e017-45c1-be76-08a290a31d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:39.256018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc220400-e017-45c1-be76-08a290a31d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:39.256303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc220400-e017-45c1-be76-08a290a31d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:39.256568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc220400-e017-45c1-be76-08a290a31d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:39.257222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc220400-e017-45c1-be76-08a290a31d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:39.258923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc220400-e017-45c1-be76-08a290a31d30 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:39.271548 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1328/2626] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:39.274421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99beac53-ee14-45a3-a3ae-9ed93ed1e404 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:39.284822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99beac53-ee14-45a3-a3ae-9ed93ed1e404 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:39.290073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef2225-5b82-4813-af8c-a4e556da0afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:39.290687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef2225-5b82-4813-af8c-a4e556da0afd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:39.290963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef2225-5b82-4813-af8c-a4e556da0afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:39.291232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef2225-5b82-4813-af8c-a4e556da0afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:39.291880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef2225-5b82-4813-af8c-a4e556da0afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:39.293479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef2225-5b82-4813-af8c-a4e556da0afd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:39.309021 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1299/2627] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:43:39.309964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc220400-e017-45c1-be76-08a290a31d30 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:39.321932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc220400-e017-45c1-be76-08a290a31d30 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:39.326625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:39.327290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:39.327588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:39.327861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] PATH_INFO: `/v3/projects/aa6c2d1828394a17ab2114efd680bed4/users/fa609556dced4a8581da3670e6d92fb2/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:39.344135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6c2d1828394a17ab2114efd680bed4, user_id=fa609556dced4a8581da3670e6d92fb2, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:39.346911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:39.360842 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1329/2628] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:39 2022] PUT /identity/v3/projects/aa6c2d1828394a17ab2114efd680bed4/users/fa609556dced4a8581da3670e6d92fb2/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:43:39.372634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496e431-531c-40c5-a877-0b9e1c277696 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:39.372915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496e431-531c-40c5-a877-0b9e1c277696 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:39.373212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496e431-531c-40c5-a877-0b9e1c277696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:39.407117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1496e431-531c-40c5-a877-0b9e1c277696 None None] MFA Rules not processed for user `fa609556dced4a8581da3670e6d92fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:39.408854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1496e431-531c-40c5-a877-0b9e1c277696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:39.484726 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1300/2629] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:39.495918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:39.497677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:39.553559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddf833-f2e6-4d40-ad97-1018c1e34e50 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:39.557153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76374da0-06ff-49ca-aadd-adb32f4b5a21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:39.557776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76374da0-06ff-49ca-aadd-adb32f4b5a21 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:39.558053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76374da0-06ff-49ca-aadd-adb32f4b5a21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:39.558319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76374da0-06ff-49ca-aadd-adb32f4b5a21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:39.566401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76374da0-06ff-49ca-aadd-adb32f4b5a21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:39.568420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76374da0-06ff-49ca-aadd-adb32f4b5a21 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:39.576960 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1330/2630] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:42.146060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896e878-2ba2-4c27-9049-0014e425c91a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:42.146623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896e878-2ba2-4c27-9049-0014e425c91a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:42.146623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896e878-2ba2-4c27-9049-0014e425c91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:42.165598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8896e878-2ba2-4c27-9049-0014e425c91a None None] MFA Rules not processed for user `e32778edb0e943f48e5f40453c97322b`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:42.166811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8896e878-2ba2-4c27-9049-0014e425c91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:42.211104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:42.223429 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1301/2631] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:43:42.688045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76374da0-06ff-49ca-aadd-adb32f4b5a21 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:42.759802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76374da0-06ff-49ca-aadd-adb32f4b5a21 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:42.764351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:42.764974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:42.765253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:42.765524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] PATH_INFO: `/v3/users/fa609556dced4a8581da3670e6d92fb2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:42.767049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa609556dced4a8581da3670e6d92fb2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:42.769110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:42.772967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:42.822979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa609556dced4a8581da3670e6d92fb2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:42.838417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa609556dced4a8581da3670e6d92fb2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:42.839272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa609556dced4a8581da3670e6d92fb2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:42.840006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa609556dced4a8581da3670e6d92fb2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:42.852224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa609556dced4a8581da3670e6d92fb2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:42.872740 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1331/2632] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:43:42 2022] DELETE /identity/v3/users/fa609556dced4a8581da3670e6d92fb2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:42.877455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736cc1-1670-4afb-b946-ce91b4ad9d54 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:42.877774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736cc1-1670-4afb-b946-ce91b4ad9d54 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:42.878047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736cc1-1670-4afb-b946-ce91b4ad9d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:42.899278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-de736cc1-1670-4afb-b946-ce91b4ad9d54 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:42.900794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-de736cc1-1670-4afb-b946-ce91b4ad9d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:42.965670 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1302/2633] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:42.974851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:42.977220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:43.053332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0226f2-67e0-456c-a791-84bc71b64c11 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:43.057308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e49c8-f777-456f-b635-289ace7b2054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:43.058071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e49c8-f777-456f-b635-289ace7b2054 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.058412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e49c8-f777-456f-b635-289ace7b2054 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.058759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e49c8-f777-456f-b635-289ace7b2054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.068559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e49c8-f777-456f-b635-289ace7b2054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:43.070129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e49c8-f777-456f-b635-289ace7b2054 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:43.080509 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1332/2634] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:43.374784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de736cc1-1670-4afb-b946-ce91b4ad9d54 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:43.376882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-de736cc1-1670-4afb-b946-ce91b4ad9d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:43.391395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de736cc1-1670-4afb-b946-ce91b4ad9d54 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:43.396693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:43.397274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.397561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.397898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] PATH_INFO: `/v3/projects/aa6c2d1828394a17ab2114efd680bed4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.399090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa6c2d1828394a17ab2114efd680bed4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:43.401013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:43.451151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa6c2d1828394a17ab2114efd680bed4', 'request_id': 'req-72fbbccd-9979-4598-b205-659495f5ffe4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:43.459568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa6c2d1828394a17ab2114efd680bed4', 'request_id': 'req-72fbbccd-9979-4598-b205-659495f5ffe4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:43.466048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa6c2d1828394a17ab2114efd680bed4', 'request_id': 'req-72fbbccd-9979-4598-b205-659495f5ffe4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:43.476588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] The token cache is being invalidate because project aa6c2d1828394a17ab2114efd680bed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:43:43.476880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:43.477544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-72fbbccd-9979-4598-b205-659495f5ffe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:43:43.479068 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1303/2635] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:43:43 2022] DELETE /identity/v3/projects/aa6c2d1828394a17ab2114efd680bed4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:43.484771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c56f0-b154-43f1-a854-2a1c5c9e62e9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.485135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c56f0-b154-43f1-a854-2a1c5c9e62e9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.485517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c56f0-b154-43f1-a854-2a1c5c9e62e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.506192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fa0c56f0-b154-43f1-a854-2a1c5c9e62e9 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:43.507558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fa0c56f0-b154-43f1-a854-2a1c5c9e62e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:43.571515 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1333/2636] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:43.579303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0203f-7ba8-447a-a3ef-ddf01bd796a8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.579588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0203f-7ba8-447a-a3ef-ddf01bd796a8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.579863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0203f-7ba8-447a-a3ef-ddf01bd796a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.600810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-27a0203f-7ba8-447a-a3ef-ddf01bd796a8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:43.602057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-27a0203f-7ba8-447a-a3ef-ddf01bd796a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:43.610121 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1304/2637] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:43:43.614405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c56f0-b154-43f1-a854-2a1c5c9e62e9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:43.624483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c56f0-b154-43f1-a854-2a1c5c9e62e9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:43.629287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:43.630450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.630724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.630994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.631639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:43:43.631946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:43.633559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:43.649618 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1334/2638] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:43:43.650919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0203f-7ba8-447a-a3ef-ddf01bd796a8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:43.660619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0203f-7ba8-447a-a3ef-ddf01bd796a8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:43.665264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaca14-fdb1-4cde-ae35-72736c44bd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:43.665880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaca14-fdb1-4cde-ae35-72736c44bd2b admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.666177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaca14-fdb1-4cde-ae35-72736c44bd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.666444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaca14-fdb1-4cde-ae35-72736c44bd2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.667065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aaca14-fdb1-4cde-ae35-72736c44bd2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:43.668938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aaca14-fdb1-4cde-ae35-72736c44bd2b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:43.696862 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1305/2639] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:43:43 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:43.701047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:43.712287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720814c-0b0d-4b03-8f38-f01a72cd1026 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:43.720180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:43.770238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8836476-d26b-4182-834e-2c08ea5b5815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:43.771430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8836476-d26b-4182-834e-2c08ea5b5815 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.771703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8836476-d26b-4182-834e-2c08ea5b5815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.771970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8836476-d26b-4182-834e-2c08ea5b5815 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.772599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8836476-d26b-4182-834e-2c08ea5b5815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:43.774543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8836476-d26b-4182-834e-2c08ea5b5815 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:43.795218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1335/2640] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:43:43 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:43:43.799381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaca14-fdb1-4cde-ae35-72736c44bd2b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:43.810756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaca14-fdb1-4cde-ae35-72736c44bd2b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:43.815381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbcdd5-f522-4c95-951c-20f6300f14f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:43.815972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbcdd5-f522-4c95-951c-20f6300f14f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.816250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbcdd5-f522-4c95-951c-20f6300f14f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.816520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbcdd5-f522-4c95-951c-20f6300f14f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.817194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbcdd5-f522-4c95-951c-20f6300f14f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:43.819881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbcdd5-f522-4c95-951c-20f6300f14f0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:43.829678 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1306/2641] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:43:43.833930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8836476-d26b-4182-834e-2c08ea5b5815 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:43.843921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8836476-d26b-4182-834e-2c08ea5b5815 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:43.848977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f4d05-1655-41e0-af0b-35afe168ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:43.849557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4d05-1655-41e0-af0b-35afe168ba68 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.849893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4d05-1655-41e0-af0b-35afe168ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.850165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4d05-1655-41e0-af0b-35afe168ba68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.850805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f4d05-1655-41e0-af0b-35afe168ba68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:43.852395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f4d05-1655-41e0-af0b-35afe168ba68 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:43.862860 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1336/2642] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:43:43.867338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbcdd5-f522-4c95-951c-20f6300f14f0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:43.877149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbcdd5-f522-4c95-951c-20f6300f14f0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:43.882733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:43.883482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.883765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.884047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] PATH_INFO: `/v3/projects/8b64dbb5554f44ca9acfaa538be5fc0b/users/5a655da92c3444f6814c37d647f88421/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.899588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b64dbb5554f44ca9acfaa538be5fc0b, user_id=5a655da92c3444f6814c37d647f88421, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:43.902356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:43.915020 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1307/2643] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:43:43 2022] PUT /identity/v3/projects/8b64dbb5554f44ca9acfaa538be5fc0b/users/5a655da92c3444f6814c37d647f88421/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:43:43.925351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14875b-7fb9-4f42-a287-d58207890c87 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:43.925672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14875b-7fb9-4f42-a287-d58207890c87 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:43.925983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14875b-7fb9-4f42-a287-d58207890c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:43.958274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7f14875b-7fb9-4f42-a287-d58207890c87 None None] MFA Rules not processed for user `5a655da92c3444f6814c37d647f88421`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:43.959592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7f14875b-7fb9-4f42-a287-d58207890c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:44.040004 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1337/2644] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:43:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:43:44.053244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:44.055237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:44.075514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:44.138935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d141-fc0d-46e8-b27f-c8ac6651fb74 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:44.147888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fe7a090bd7941bb827d301aa4949857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:44.148534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:44.148804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:44.149077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:44.159314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:44.160712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:44.185852 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1308/2645] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:43:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:43:54.033865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:54.168990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=20 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:43:58.791806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d4656-adb2-4954-8f0e-3e2d29d0f4bc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:58.792317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d4656-adb2-4954-8f0e-3e2d29d0f4bc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:58.792424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d4656-adb2-4954-8f0e-3e2d29d0f4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:58.832450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-533d4656-adb2-4954-8f0e-3e2d29d0f4bc None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:43:58.833676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-533d4656-adb2-4954-8f0e-3e2d29d0f4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:58.913107 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1338/2646] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:43:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:43:58.960097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:43:58.961499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:43:59.060788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5095b7-c7f0-412d-9d9e-4752bcd32cd6 service glance] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:43:59.065846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:43:59.066453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:43:59.066731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:43:59.067000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:43:59.075704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:43:59.077293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:43:59.085194 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1309/2647] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:43:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:44:08.911340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=45 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:09.082212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=46 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:15.684967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d4656-adb2-4954-8f0e-3e2d29d0f4bc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:44:15.716663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:44:15.718112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:15.786854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d4656-adb2-4954-8f0e-3e2d29d0f4bc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:44:15.791964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:44:15.792698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:15.793057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:15.793337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] PATH_INFO: `/v3/users/5a655da92c3444f6814c37d647f88421` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:15.794586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a655da92c3444f6814c37d647f88421)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:44:15.797621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:44:15.818276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca5954-9ec2-41bd-a7e1-f49a0d28f96b service swift] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:44:15.823722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a9d36-cf2f-4636-8ba1-1fed6f1ddf43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:44:15.824430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9d36-cf2f-4636-8ba1-1fed6f1ddf43 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:15.824778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9d36-cf2f-4636-8ba1-1fed6f1ddf43 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:15.825116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9d36-cf2f-4636-8ba1-1fed6f1ddf43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:15.827034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-942a9d36-cf2f-4636-8ba1-1fed6f1ddf43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:15.846806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a9d36-cf2f-4636-8ba1-1fed6f1ddf43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:44:15.848465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a9d36-cf2f-4636-8ba1-1fed6f1ddf43 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:44:15.857863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a655da92c3444f6814c37d647f88421'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:15.861973 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1310/2648] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:44:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:44:15.867918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a655da92c3444f6814c37d647f88421'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:15.868767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a655da92c3444f6814c37d647f88421'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:15.869371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a655da92c3444f6814c37d647f88421'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:15.879005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a655da92c3444f6814c37d647f88421'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:15.905287 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1339/2649] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:44:15 2022] DELETE /identity/v3/users/5a655da92c3444f6814c37d647f88421 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:44:15.907885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fc33-55ab-406a-8a18-ff0a9115a231 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:15.908515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fc33-55ab-406a-8a18-ff0a9115a231 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:15.909072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62fc33-55ab-406a-8a18-ff0a9115a231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:15.960039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bc62fc33-55ab-406a-8a18-ff0a9115a231 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:44:15.961570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bc62fc33-55ab-406a-8a18-ff0a9115a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:16.032868 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1311/2650] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:44:16.046340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:44:16.048834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:16.122696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:16.134176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b362d-5a54-43ae-aba6-efe852ef1af1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:44:16.138685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992f14f-eb16-45bc-9d0a-81c255453ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:44:16.139267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992f14f-eb16-45bc-9d0a-81c255453ccf service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:16.139569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992f14f-eb16-45bc-9d0a-81c255453ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:16.139841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992f14f-eb16-45bc-9d0a-81c255453ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:16.150233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992f14f-eb16-45bc-9d0a-81c255453ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:44:16.151782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992f14f-eb16-45bc-9d0a-81c255453ccf service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:44:16.161981 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1340/2651] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:44:16.447004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fc33-55ab-406a-8a18-ff0a9115a231 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:44:16.448432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bc62fc33-55ab-406a-8a18-ff0a9115a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:16.461415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62fc33-55ab-406a-8a18-ff0a9115a231 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:44:16.466496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:44:16.467126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:16.467433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:16.467711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] PATH_INFO: `/v3/projects/8b64dbb5554f44ca9acfaa538be5fc0b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:16.469483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b64dbb5554f44ca9acfaa538be5fc0b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:44:16.471421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:44:16.519428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:16.521001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b64dbb5554f44ca9acfaa538be5fc0b', 'request_id': 'req-877ead56-ca03-4a06-a75a-01ba334393d1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:16.528508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b64dbb5554f44ca9acfaa538be5fc0b', 'request_id': 'req-877ead56-ca03-4a06-a75a-01ba334393d1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:16.535357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b64dbb5554f44ca9acfaa538be5fc0b', 'request_id': 'req-877ead56-ca03-4a06-a75a-01ba334393d1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:16.545593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] The token cache is being invalidate because project 8b64dbb5554f44ca9acfaa538be5fc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:44:16.545969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:16.547261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-877ead56-ca03-4a06-a75a-01ba334393d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:44:16.548853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1312/2652] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:44:16 2022] DELETE /identity/v3/projects/8b64dbb5554f44ca9acfaa538be5fc0b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:44:17.675635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b769-cd1f-440f-a053-82f203e6d998 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:17.676142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b769-cd1f-440f-a053-82f203e6d998 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:17.676241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b769-cd1f-440f-a053-82f203e6d998 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:17.677591 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1341/2653] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 30 09:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 30 09:44:17.682727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1981afe-04de-4299-9073-9b9ee93a7165 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:17.683229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1981afe-04de-4299-9073-9b9ee93a7165 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:17.683333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1981afe-04de-4299-9073-9b9ee93a7165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:17.726464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b1981afe-04de-4299-9073-9b9ee93a7165 None None] MFA Rules not processed for user `aa055ee122ce4e92b4da7f8222521a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:44:17.727683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b1981afe-04de-4299-9073-9b9ee93a7165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:17.806026 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1313/2654] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 30 09:44:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:44:17.818561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95b769-cd1f-440f-a053-82f203e6d998 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:44:17.820286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4c95b769-cd1f-440f-a053-82f203e6d998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:17.878264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95b769-cd1f-440f-a053-82f203e6d998 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:44:17.881860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fe7a090bd7941bb827d301aa4949857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:44:17.882459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:17.882740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:17.883011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:17.893538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:44:17.895011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:44:17.905545 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1342/2655] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:44:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:44:19.840035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2526358-20da-4907-957a-056f8ef708a5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:19.840533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2526358-20da-4907-957a-056f8ef708a5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:19.840631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2526358-20da-4907-957a-056f8ef708a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:19.860757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e2526358-20da-4907-957a-056f8ef708a5 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:44:19.861982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e2526358-20da-4907-957a-056f8ef708a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:19.916317 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1314/2656] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:44:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:44:19.928961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:44:19.930396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:19.998634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff448a15-dba2-4a6f-8270-c4b8641d7040 service glance] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:44:20.003951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea965db-ad82-4f86-bc8c-08e8aae01304 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:44:20.004541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea965db-ad82-4f86-bc8c-08e8aae01304 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:20.004818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea965db-ad82-4f86-bc8c-08e8aae01304 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:20.005088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea965db-ad82-4f86-bc8c-08e8aae01304 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:20.013595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea965db-ad82-4f86-bc8c-08e8aae01304 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:44:20.015667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea965db-ad82-4f86-bc8c-08e8aae01304 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:44:20.023710 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1343/2657] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:44:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:44:29.909952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=76,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:30.020402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=65 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:40.274425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942243e3-c5cb-4199-9769-bf2b4159cb0f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:40.274873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942243e3-c5cb-4199-9769-bf2b4159cb0f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:40.274981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942243e3-c5cb-4199-9769-bf2b4159cb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:40.295127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-942243e3-c5cb-4199-9769-bf2b4159cb0f None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:44:40.296322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-942243e3-c5cb-4199-9769-bf2b4159cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:40.303957 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1315/2658] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 30 09:44:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:44:40.317137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea965db-ad82-4f86-bc8c-08e8aae01304 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:44:40.328258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea965db-ad82-4f86-bc8c-08e8aae01304 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:44:40.333616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91b273-d0ff-490d-9f6a-54a934665d7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:44:40.334230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91b273-d0ff-490d-9f6a-54a934665d7e service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:40.334506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91b273-d0ff-490d-9f6a-54a934665d7e service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:40.334773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91b273-d0ff-490d-9f6a-54a934665d7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:40.343033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91b273-d0ff-490d-9f6a-54a934665d7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:44:40.344689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91b273-d0ff-490d-9f6a-54a934665d7e service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:44:40.352917 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1344/2659] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:44:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:44:50.288835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:50.349506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:44:58.194574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942243e3-c5cb-4199-9769-bf2b4159cb0f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:44:58.196032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-942243e3-c5cb-4199-9769-bf2b4159cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:58.250153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942243e3-c5cb-4199-9769-bf2b4159cb0f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:44:58.253860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:44:58.254485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:44:58.254759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:44:58.255026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:44:58.256732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:44:58.316176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:44:58.318252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:44:58.327210 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1316/2660] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:44:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:04.349658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91b273-d0ff-490d-9f6a-54a934665d7e service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:04.358889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91b273-d0ff-490d-9f6a-54a934665d7e service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:04.362599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:04.363181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:04.363454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:04.363720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:04.365462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:04.429587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:04.431080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:04.440750 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1345/2661] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:08.323005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=48 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:10.487227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:10.490824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:10.506596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c97dc-fe5c-4864-815c-d6bc79145bd8 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:10.511174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:10.511805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:10.512085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:10.512357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] PATH_INFO: `/v3/users/468fa04c3a11433aa8f6127b72b36c0c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:10.513485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=468fa04c3a11433aa8f6127b72b36c0c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:10.515694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:10.564283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468fa04c3a11433aa8f6127b72b36c0c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.571208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468fa04c3a11433aa8f6127b72b36c0c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.571900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468fa04c3a11433aa8f6127b72b36c0c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.572518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468fa04c3a11433aa8f6127b72b36c0c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.581202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468fa04c3a11433aa8f6127b72b36c0c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.599172 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1317/2662] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:10 2022] DELETE /identity/v3/users/468fa04c3a11433aa8f6127b72b36c0c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:10.769307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:10.780010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46def2-edc6-48bd-8417-7c66e4cdd6ec service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:10.839011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:10.839566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:10.839845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:10.840117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] PATH_INFO: `/v3/projects/2c1cb4c57c924a24ab3368be3a3384cb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:10.841217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1cb4c57c924a24ab3368be3a3384cb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:10.843246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:10.889310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1cb4c57c924a24ab3368be3a3384cb', 'request_id': 'req-76111990-7ca2-4fd8-b62f-0130328100dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.897374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1cb4c57c924a24ab3368be3a3384cb', 'request_id': 'req-76111990-7ca2-4fd8-b62f-0130328100dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.903524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1cb4c57c924a24ab3368be3a3384cb', 'request_id': 'req-76111990-7ca2-4fd8-b62f-0130328100dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.913872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] The token cache is being invalidate because project 2c1cb4c57c924a24ab3368be3a3384cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:10.914163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.914797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:10.916277 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1346/2663] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:10 2022] DELETE /identity/v3/projects/2c1cb4c57c924a24ab3368be3a3384cb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:10.920293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:10.921748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:10.934820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4bbf2-d546-4fff-aa0c-ccb61f1d98ec admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:10.991662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:10.992625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:10.992625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:10.992924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] PATH_INFO: `/v3/users/a7c72a2db99b4fe2a9c726c10bbe3af9` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:10.994198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7c72a2db99b4fe2a9c726c10bbe3af9)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:10.996098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:11.051957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7c72a2db99b4fe2a9c726c10bbe3af9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.059093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7c72a2db99b4fe2a9c726c10bbe3af9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.059785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7c72a2db99b4fe2a9c726c10bbe3af9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.060392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7c72a2db99b4fe2a9c726c10bbe3af9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.068238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7c72a2db99b4fe2a9c726c10bbe3af9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.078086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:11.092443 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1318/2664] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:10 2022] DELETE /identity/v3/users/a7c72a2db99b4fe2a9c726c10bbe3af9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:11.270135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:11.271644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:11.285402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76111990-7ca2-4fd8-b62f-0130328100dc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:11.310964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:11.346054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:11.346617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:11.346898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:11.347171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] PATH_INFO: `/v3/projects/39a6ad090122467da59af411d4ea3551` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:11.348340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a6ad090122467da59af411d4ea3551)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:11.350308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:11.396419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a6ad090122467da59af411d4ea3551', 'request_id': 'req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.404027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a6ad090122467da59af411d4ea3551', 'request_id': 'req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.410973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a6ad090122467da59af411d4ea3551', 'request_id': 'req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.421052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] The token cache is being invalidate because project 39a6ad090122467da59af411d4ea3551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:11.421338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.422016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:11.424103 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1347/2665] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:11 2022] DELETE /identity/v3/projects/39a6ad090122467da59af411d4ea3551 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:21.091445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:21.418095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:30.113353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:30.114789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:30.173138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b7462-5a99-4d43-a4bc-c7bb684472bf admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:30.220494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:30.221103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:30.221377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:30.221671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:30.223376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:30.284718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:30.286097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:30.295584 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1319/2666] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:33.427977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:33.429408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:33.443613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209a775-4a4b-4bd1-9d6d-1c7bea0d8ee5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:33.448168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:33.448793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:33.449076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:33.449350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] PATH_INFO: `/v3/users/e32778edb0e943f48e5f40453c97322b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:33.450502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32778edb0e943f48e5f40453c97322b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:33.452467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:33.497593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32778edb0e943f48e5f40453c97322b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.505182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32778edb0e943f48e5f40453c97322b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.505864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32778edb0e943f48e5f40453c97322b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.506463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32778edb0e943f48e5f40453c97322b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.513853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ca73a77c-099b-4bf8-a1d4-1cb669f2aa38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32778edb0e943f48e5f40453c97322b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.531918 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1348/2667] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:33 2022] DELETE /identity/v3/users/e32778edb0e943f48e5f40453c97322b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:33.705540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:33.723442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe6e67-ad90-4c9b-a7d7-f142c31acaf9 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:33.781436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:33.782090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:33.782366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:33.782638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] PATH_INFO: `/v3/projects/277b0f36e38f42909ad307ca1d09b7da` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:33.783760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=277b0f36e38f42909ad307ca1d09b7da)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:33.785667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:33.793382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:33.834477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277b0f36e38f42909ad307ca1d09b7da', 'request_id': 'req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.841320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277b0f36e38f42909ad307ca1d09b7da', 'request_id': 'req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.848199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277b0f36e38f42909ad307ca1d09b7da', 'request_id': 'req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.858189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] The token cache is being invalidate because project 277b0f36e38f42909ad307ca1d09b7da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:33.858475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.859991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1be09ab-b7ee-4a68-b1e0-7a502ae91abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:33.861481 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1320/2668] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:33 2022] DELETE /identity/v3/projects/277b0f36e38f42909ad307ca1d09b7da => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:33.866398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22287b70-95ed-4663-b569-166c7a5c4345 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:33.866688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22287b70-95ed-4663-b569-166c7a5c4345 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:33.866964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22287b70-95ed-4663-b569-166c7a5c4345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:33.904767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-22287b70-95ed-4663-b569-166c7a5c4345 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:33.906650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-22287b70-95ed-4663-b569-166c7a5c4345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:33.966617 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1349/2669] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:33.975067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b67e-beba-4282-a1a6-ed322e97cdff None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:33.975351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b67e-beba-4282-a1a6-ed322e97cdff None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:33.975624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482b67e-beba-4282-a1a6-ed322e97cdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:33.995530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9482b67e-beba-4282-a1a6-ed322e97cdff None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:33.996725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9482b67e-beba-4282-a1a6-ed322e97cdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:34.004539 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1321/2670] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:34.008756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22287b70-95ed-4663-b569-166c7a5c4345 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.019277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22287b70-95ed-4663-b569-166c7a5c4345 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.023936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.024526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.024808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.025078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.025789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:45:34.026091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.027696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.039734 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1350/2671] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:45:34.044005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482b67e-beba-4282-a1a6-ed322e97cdff None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.053815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482b67e-beba-4282-a1a6-ed322e97cdff None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.058412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dac6d3-f6d2-4cd9-81a4-e097b8995d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.058998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dac6d3-f6d2-4cd9-81a4-e097b8995d2e admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.059275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dac6d3-f6d2-4cd9-81a4-e097b8995d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.059545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dac6d3-f6d2-4cd9-81a4-e097b8995d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.060157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dac6d3-f6d2-4cd9-81a4-e097b8995d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.062082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dac6d3-f6d2-4cd9-81a4-e097b8995d2e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.089179 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1322/2672] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:34 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:34.094186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.104128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b4f23-20c1-4d29-af6b-2d59ae58627a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.146376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:34.163191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffead22-da1d-4a42-a61a-7e7dd37a3e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.163724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffead22-da1d-4a42-a61a-7e7dd37a3e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.163999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffead22-da1d-4a42-a61a-7e7dd37a3e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.164266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffead22-da1d-4a42-a61a-7e7dd37a3e30 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.164893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffead22-da1d-4a42-a61a-7e7dd37a3e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.166843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffead22-da1d-4a42-a61a-7e7dd37a3e30 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.188395 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1351/2673] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:34 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:34.192635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dac6d3-f6d2-4cd9-81a4-e097b8995d2e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.203289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dac6d3-f6d2-4cd9-81a4-e097b8995d2e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.207881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6a482-9bf3-473d-bb57-a68a050b96cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.208465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6a482-9bf3-473d-bb57-a68a050b96cb admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.208740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6a482-9bf3-473d-bb57-a68a050b96cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.209009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6a482-9bf3-473d-bb57-a68a050b96cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.209655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6a482-9bf3-473d-bb57-a68a050b96cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.211246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6a482-9bf3-473d-bb57-a68a050b96cb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.221332 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1323/2674] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:34.225295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffead22-da1d-4a42-a61a-7e7dd37a3e30 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.235613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffead22-da1d-4a42-a61a-7e7dd37a3e30 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.240208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5171d-0989-4086-8781-5a138e46ca72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.240798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5171d-0989-4086-8781-5a138e46ca72 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.241083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5171d-0989-4086-8781-5a138e46ca72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.241354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5171d-0989-4086-8781-5a138e46ca72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.242024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5171d-0989-4086-8781-5a138e46ca72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.243623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5171d-0989-4086-8781-5a138e46ca72 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.254315 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1352/2675] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:34.258524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6a482-9bf3-473d-bb57-a68a050b96cb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.269465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6a482-9bf3-473d-bb57-a68a050b96cb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.274114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e847e-35dc-44f5-b172-418325ff32be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.274792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e847e-35dc-44f5-b172-418325ff32be admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.275067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e847e-35dc-44f5-b172-418325ff32be admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.275336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e847e-35dc-44f5-b172-418325ff32be admin admin] PATH_INFO: `/v3/projects/aab31972d34b46cbad3136b95fb4ebea/users/cf738d2612e9464ab0d19dd77f434a7a/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.290456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e847e-35dc-44f5-b172-418325ff32be admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab31972d34b46cbad3136b95fb4ebea, user_id=cf738d2612e9464ab0d19dd77f434a7a, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.293085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e847e-35dc-44f5-b172-418325ff32be admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.305730 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1324/2676] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:34 2022] PUT /identity/v3/projects/aab31972d34b46cbad3136b95fb4ebea/users/cf738d2612e9464ab0d19dd77f434a7a/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:34.316080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c994a89-0fc9-4f93-aec0-f9462cdad462 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.316397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c994a89-0fc9-4f93-aec0-f9462cdad462 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.316674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c994a89-0fc9-4f93-aec0-f9462cdad462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.345214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0c994a89-0fc9-4f93-aec0-f9462cdad462 None None] MFA Rules not processed for user `cf738d2612e9464ab0d19dd77f434a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:34.346464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c994a89-0fc9-4f93-aec0-f9462cdad462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:34.411178 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1353/2677] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:34.415718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e847e-35dc-44f5-b172-418325ff32be admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.477131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e847e-35dc-44f5-b172-418325ff32be admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.481792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207afc81-bdb0-4a0f-a147-c3e034c9f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.482392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207afc81-bdb0-4a0f-a147-c3e034c9f887 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.482673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207afc81-bdb0-4a0f-a147-c3e034c9f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.482945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207afc81-bdb0-4a0f-a147-c3e034c9f887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.483580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207afc81-bdb0-4a0f-a147-c3e034c9f887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.485502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207afc81-bdb0-4a0f-a147-c3e034c9f887 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.511268 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1325/2678] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:34 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:34.516270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c994a89-0fc9-4f93-aec0-f9462cdad462 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.526588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c994a89-0fc9-4f93-aec0-f9462cdad462 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.583857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf08a-0a47-4a92-9adb-39300da9d2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.584467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf08a-0a47-4a92-9adb-39300da9d2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.584740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf08a-0a47-4a92-9adb-39300da9d2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.585027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bf08a-0a47-4a92-9adb-39300da9d2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.585783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bf08a-0a47-4a92-9adb-39300da9d2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.587735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bf08a-0a47-4a92-9adb-39300da9d2e9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.608475 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1354/2679] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:34 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:45:34.612903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207afc81-bdb0-4a0f-a147-c3e034c9f887 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.622842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207afc81-bdb0-4a0f-a147-c3e034c9f887 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.628030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9250c2-e5d6-4b98-b17f-962e10bb0b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.628619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9250c2-e5d6-4b98-b17f-962e10bb0b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.628895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9250c2-e5d6-4b98-b17f-962e10bb0b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.629163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9250c2-e5d6-4b98-b17f-962e10bb0b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.629829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9250c2-e5d6-4b98-b17f-962e10bb0b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.631459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9250c2-e5d6-4b98-b17f-962e10bb0b1f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.642571 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1326/2680] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:34.646664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf08a-0a47-4a92-9adb-39300da9d2e9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.656954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bf08a-0a47-4a92-9adb-39300da9d2e9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.661815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5444d5-fdd3-4e09-a490-28499e37f9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.662512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5444d5-fdd3-4e09-a490-28499e37f9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.662803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5444d5-fdd3-4e09-a490-28499e37f9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.663202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5444d5-fdd3-4e09-a490-28499e37f9fe admin admin] PATH_INFO: `/v3/projects/4f21c0c8fe0d4291817dce2ca58298ba/users/212a862de4094bf7acc2e13f302563bf/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.677841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5444d5-fdd3-4e09-a490-28499e37f9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f21c0c8fe0d4291817dce2ca58298ba, user_id=212a862de4094bf7acc2e13f302563bf, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.680427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5444d5-fdd3-4e09-a490-28499e37f9fe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.694616 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1355/2681] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:34 2022] PUT /identity/v3/projects/4f21c0c8fe0d4291817dce2ca58298ba/users/212a862de4094bf7acc2e13f302563bf/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:34.698580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9250c2-e5d6-4b98-b17f-962e10bb0b1f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.709069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9250c2-e5d6-4b98-b17f-962e10bb0b1f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.712624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:34.766394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67c2f3-949c-4fed-896e-7e991666613c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.766993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67c2f3-949c-4fed-896e-7e991666613c admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.767270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67c2f3-949c-4fed-896e-7e991666613c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.767538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67c2f3-949c-4fed-896e-7e991666613c admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.768174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67c2f3-949c-4fed-896e-7e991666613c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.769896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67c2f3-949c-4fed-896e-7e991666613c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.779246 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1327/2682] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:34.783472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5444d5-fdd3-4e09-a490-28499e37f9fe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.793579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5444d5-fdd3-4e09-a490-28499e37f9fe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:34.798417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:34.799055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.799336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.799610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] PATH_INFO: `/v3/projects/4f21c0c8fe0d4291817dce2ca58298ba/users/212a862de4094bf7acc2e13f302563bf/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.801510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f21c0c8fe0d4291817dce2ca58298ba, user_id=212a862de4094bf7acc2e13f302563bf, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:34.804184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:34.814835 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1356/2683] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:34 2022] PUT /identity/v3/projects/4f21c0c8fe0d4291817dce2ca58298ba/users/212a862de4094bf7acc2e13f302563bf/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:34.826360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa23042-d2a9-43b0-816f-88e89ae05494 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:34.826674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa23042-d2a9-43b0-816f-88e89ae05494 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:34.826950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa23042-d2a9-43b0-816f-88e89ae05494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:34.856527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-eaa23042-d2a9-43b0-816f-88e89ae05494 None None] MFA Rules not processed for user `212a862de4094bf7acc2e13f302563bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:34.857876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eaa23042-d2a9-43b0-816f-88e89ae05494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:34.925566 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1328/2684] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:34.936802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:34.938724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:34.943478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:35.018290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd0edd-e98a-43b1-8a32-b341777f184a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:35.021996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ab0fc-3438-4c40-b039-eb40a0556742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:35.022606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ab0fc-3438-4c40-b039-eb40a0556742 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:35.022886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ab0fc-3438-4c40-b039-eb40a0556742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:35.023156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ab0fc-3438-4c40-b039-eb40a0556742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:35.033352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ab0fc-3438-4c40-b039-eb40a0556742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:35.034858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ab0fc-3438-4c40-b039-eb40a0556742 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:35.044483 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1357/2685] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:35.472198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa23042-d2a9-43b0-816f-88e89ae05494 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:35.535356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa23042-d2a9-43b0-816f-88e89ae05494 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:35.539837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:35.540468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:35.540744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:35.541015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] PATH_INFO: `/v3/users/cf738d2612e9464ab0d19dd77f434a7a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:35.542277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf738d2612e9464ab0d19dd77f434a7a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:35.544324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:35.576552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:35.590866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf738d2612e9464ab0d19dd77f434a7a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:35.598169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf738d2612e9464ab0d19dd77f434a7a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:35.598825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf738d2612e9464ab0d19dd77f434a7a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:35.599432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf738d2612e9464ab0d19dd77f434a7a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:35.607415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf738d2612e9464ab0d19dd77f434a7a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:35.625394 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1329/2686] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:35 2022] DELETE /identity/v3/users/cf738d2612e9464ab0d19dd77f434a7a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:35.629877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddbee1-fc22-418d-b148-5214324181f1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:35.630388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddbee1-fc22-418d-b148-5214324181f1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:35.630586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddbee1-fc22-418d-b148-5214324181f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:35.650054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bdddbee1-fc22-418d-b148-5214324181f1 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:35.651262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bdddbee1-fc22-418d-b148-5214324181f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:35.709432 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1358/2687] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:35.719387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:35.720765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:35.777993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76257cb4-df09-4c82-aad3-1407d95e663a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:35.781944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0098c0-7805-429e-aaed-5dce98d8d680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:35.782544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0098c0-7805-429e-aaed-5dce98d8d680 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:35.782823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0098c0-7805-429e-aaed-5dce98d8d680 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:35.783100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0098c0-7805-429e-aaed-5dce98d8d680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:35.791916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0098c0-7805-429e-aaed-5dce98d8d680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:35.793337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0098c0-7805-429e-aaed-5dce98d8d680 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:35.803413 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1330/2688] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:36.067807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdddbee1-fc22-418d-b148-5214324181f1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:36.069615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bdddbee1-fc22-418d-b148-5214324181f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:36.083260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdddbee1-fc22-418d-b148-5214324181f1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:36.087888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:36.088518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:36.088799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:36.089072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] PATH_INFO: `/v3/projects/aab31972d34b46cbad3136b95fb4ebea` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:36.090222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab31972d34b46cbad3136b95fb4ebea)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:36.092183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:36.141208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab31972d34b46cbad3136b95fb4ebea', 'request_id': 'req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.148533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab31972d34b46cbad3136b95fb4ebea', 'request_id': 'req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.154670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab31972d34b46cbad3136b95fb4ebea', 'request_id': 'req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.164700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] The token cache is being invalidate because project aab31972d34b46cbad3136b95fb4ebea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:36.164985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.165596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.167078 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1359/2689] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:36 2022] DELETE /identity/v3/projects/aab31972d34b46cbad3136b95fb4ebea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:36.171061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0098c0-7805-429e-aaed-5dce98d8d680 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:36.180820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0098c0-7805-429e-aaed-5dce98d8d680 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:36.236233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:36.236856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:36.237128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:36.237395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] PATH_INFO: `/v3/users/212a862de4094bf7acc2e13f302563bf` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:36.238534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=212a862de4094bf7acc2e13f302563bf)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:36.240777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:36.287186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212a862de4094bf7acc2e13f302563bf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.291474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:36.295550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212a862de4094bf7acc2e13f302563bf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.296227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212a862de4094bf7acc2e13f302563bf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.297195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212a862de4094bf7acc2e13f302563bf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.305368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b899f48d-e5c0-4add-9755-fd199b18f1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212a862de4094bf7acc2e13f302563bf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.323291 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1331/2690] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:36 2022] DELETE /identity/v3/users/212a862de4094bf7acc2e13f302563bf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:36.596801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:36.598360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:36.614098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d288ad6-a005-401e-bdf4-dbdce35ed5c6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:36.623185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:36.675427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:36.676129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:36.676340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:36.676612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] PATH_INFO: `/v3/projects/4f21c0c8fe0d4291817dce2ca58298ba` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:36.677774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f21c0c8fe0d4291817dce2ca58298ba)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:36.679683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:36.724482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f21c0c8fe0d4291817dce2ca58298ba', 'request_id': 'req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.731640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f21c0c8fe0d4291817dce2ca58298ba', 'request_id': 'req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.738160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f21c0c8fe0d4291817dce2ca58298ba', 'request_id': 'req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.748142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] The token cache is being invalidate because project 4f21c0c8fe0d4291817dce2ca58298ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:36.748426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.749055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1f3c692-ff47-4bd3-9d86-eb7f3aba5bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:36.751150 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1360/2691] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:36 2022] DELETE /identity/v3/projects/4f21c0c8fe0d4291817dce2ca58298ba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:36.755961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99f59-de7c-473e-965c-c06d6ff3ea4d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:36.756423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99f59-de7c-473e-965c-c06d6ff3ea4d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:36.756544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99f59-de7c-473e-965c-c06d6ff3ea4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:36.777348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3ea99f59-de7c-473e-965c-c06d6ff3ea4d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:36.778507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3ea99f59-de7c-473e-965c-c06d6ff3ea4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:36.834753 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1332/2692] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:36.842578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c0a5d-c8c2-480e-94e9-b932b084b67f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:36.842867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c0a5d-c8c2-480e-94e9-b932b084b67f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:36.843138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c0a5d-c8c2-480e-94e9-b932b084b67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:36.863338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-586c0a5d-c8c2-480e-94e9-b932b084b67f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:36.864549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-586c0a5d-c8c2-480e-94e9-b932b084b67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:36.872445 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1361/2693] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:45:36.877479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99f59-de7c-473e-965c-c06d6ff3ea4d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:36.887370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99f59-de7c-473e-965c-c06d6ff3ea4d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:36.892813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:36.893406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:36.893725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:36.894005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:36.894648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:45:36.894956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:36.896526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:36.907986 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1333/2694] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:45:36.912179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c0a5d-c8c2-480e-94e9-b932b084b67f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:36.921341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c0a5d-c8c2-480e-94e9-b932b084b67f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:36.926339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd22a39-471e-4c0b-bf7b-b1f4493e1ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:36.926927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd22a39-471e-4c0b-bf7b-b1f4493e1ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:36.927209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd22a39-471e-4c0b-bf7b-b1f4493e1ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:36.927480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd22a39-471e-4c0b-bf7b-b1f4493e1ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:36.928107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd22a39-471e-4c0b-bf7b-b1f4493e1ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:36.930047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd22a39-471e-4c0b-bf7b-b1f4493e1ccd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:36.957916 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1362/2695] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:36 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:36.961980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:36.971939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b94fd9-e157-437e-bbbc-2c3dda3e4842 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.027301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2161c-6d33-452c-a96b-4d8564a37797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.027994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2161c-6d33-452c-a96b-4d8564a37797 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.028216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2161c-6d33-452c-a96b-4d8564a37797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.028490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2161c-6d33-452c-a96b-4d8564a37797 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.029125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2161c-6d33-452c-a96b-4d8564a37797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.031090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2161c-6d33-452c-a96b-4d8564a37797 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.051782 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1334/2696] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:36 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:37.055826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd22a39-471e-4c0b-bf7b-b1f4493e1ccd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.066815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd22a39-471e-4c0b-bf7b-b1f4493e1ccd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.071351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4d95b-e121-4b87-9ee8-4264ab621ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.071936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4d95b-e121-4b87-9ee8-4264ab621ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.072211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4d95b-e121-4b87-9ee8-4264ab621ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.072478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4d95b-e121-4b87-9ee8-4264ab621ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.073107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4d95b-e121-4b87-9ee8-4264ab621ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.074757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4d95b-e121-4b87-9ee8-4264ab621ef8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.084949 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1363/2697] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:37.088684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2161c-6d33-452c-a96b-4d8564a37797 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.098566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2161c-6d33-452c-a96b-4d8564a37797 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.103095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d25f4-0a01-4f5f-9e52-b3e0a546da03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.103680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d25f4-0a01-4f5f-9e52-b3e0a546da03 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.103959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d25f4-0a01-4f5f-9e52-b3e0a546da03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.104237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d25f4-0a01-4f5f-9e52-b3e0a546da03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.104907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d25f4-0a01-4f5f-9e52-b3e0a546da03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.106580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d25f4-0a01-4f5f-9e52-b3e0a546da03 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.116802 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1335/2698] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:37.120735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4d95b-e121-4b87-9ee8-4264ab621ef8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.131365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4d95b-e121-4b87-9ee8-4264ab621ef8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.136553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7d15a-3ba5-4430-8e1b-90a40f44e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.137211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7d15a-3ba5-4430-8e1b-90a40f44e6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.137495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7d15a-3ba5-4430-8e1b-90a40f44e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.137799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7d15a-3ba5-4430-8e1b-90a40f44e6ef admin admin] PATH_INFO: `/v3/projects/0b1339eb65394485942d6e5b3db3578b/users/0ae40cbcf39d44aa976de8dcbf63dd32/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.153227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7d15a-3ba5-4430-8e1b-90a40f44e6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1339eb65394485942d6e5b3db3578b, user_id=0ae40cbcf39d44aa976de8dcbf63dd32, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.156030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7d15a-3ba5-4430-8e1b-90a40f44e6ef admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.168171 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1364/2699] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:37 2022] PUT /identity/v3/projects/0b1339eb65394485942d6e5b3db3578b/users/0ae40cbcf39d44aa976de8dcbf63dd32/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:37.178412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce41b5-03b4-45cb-b500-684e84ecfa22 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.178720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce41b5-03b4-45cb-b500-684e84ecfa22 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.178990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce41b5-03b4-45cb-b500-684e84ecfa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.207221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-69ce41b5-03b4-45cb-b500-684e84ecfa22 None None] MFA Rules not processed for user `0ae40cbcf39d44aa976de8dcbf63dd32`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:37.208427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-69ce41b5-03b4-45cb-b500-684e84ecfa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:37.232765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:37.275320 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1336/2700] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:37.279968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7d15a-3ba5-4430-8e1b-90a40f44e6ef admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.320902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:37.343419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7d15a-3ba5-4430-8e1b-90a40f44e6ef admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.348384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de50db-7aad-4137-8ed8-e894dd60fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.348986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de50db-7aad-4137-8ed8-e894dd60fb90 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.349267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de50db-7aad-4137-8ed8-e894dd60fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.349531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de50db-7aad-4137-8ed8-e894dd60fb90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.350189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de50db-7aad-4137-8ed8-e894dd60fb90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.352095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de50db-7aad-4137-8ed8-e894dd60fb90 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.382235 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1365/2701] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:37 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:37.386690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce41b5-03b4-45cb-b500-684e84ecfa22 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.398274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce41b5-03b4-45cb-b500-684e84ecfa22 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.458922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62d116-782f-4118-b381-50d91baf9375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.459530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62d116-782f-4118-b381-50d91baf9375 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.459811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62d116-782f-4118-b381-50d91baf9375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.460079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62d116-782f-4118-b381-50d91baf9375 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.460703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62d116-782f-4118-b381-50d91baf9375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.462628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62d116-782f-4118-b381-50d91baf9375 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.491209 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1337/2702] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:37 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:37.492489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de50db-7aad-4137-8ed8-e894dd60fb90 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.503835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de50db-7aad-4137-8ed8-e894dd60fb90 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.509422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5cfb5-e842-4b48-a52f-487277677256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.510294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5cfb5-e842-4b48-a52f-487277677256 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.510777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5cfb5-e842-4b48-a52f-487277677256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.511053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5cfb5-e842-4b48-a52f-487277677256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.511712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5cfb5-e842-4b48-a52f-487277677256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.513349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5cfb5-e842-4b48-a52f-487277677256 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.524785 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1366/2703] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:37.529041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62d116-782f-4118-b381-50d91baf9375 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.539577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62d116-782f-4118-b381-50d91baf9375 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.544245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df17436-ff9c-4e54-b4fe-e078e460c061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.544895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df17436-ff9c-4e54-b4fe-e078e460c061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.545171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df17436-ff9c-4e54-b4fe-e078e460c061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.545445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df17436-ff9c-4e54-b4fe-e078e460c061 admin admin] PATH_INFO: `/v3/projects/f175d07458f6483a9e2b83d61e5293db/users/586fb3d9518341a3b543a72d05825209/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.560394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df17436-ff9c-4e54-b4fe-e078e460c061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f175d07458f6483a9e2b83d61e5293db, user_id=586fb3d9518341a3b543a72d05825209, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.563722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df17436-ff9c-4e54-b4fe-e078e460c061 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.577339 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1338/2704] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:37 2022] PUT /identity/v3/projects/f175d07458f6483a9e2b83d61e5293db/users/586fb3d9518341a3b543a72d05825209/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:37.581505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5cfb5-e842-4b48-a52f-487277677256 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.591746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5cfb5-e842-4b48-a52f-487277677256 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.657562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753a38e-98a7-405a-a6c5-cebebbe49db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.658201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753a38e-98a7-405a-a6c5-cebebbe49db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.658507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753a38e-98a7-405a-a6c5-cebebbe49db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.658774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753a38e-98a7-405a-a6c5-cebebbe49db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.659413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753a38e-98a7-405a-a6c5-cebebbe49db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.661015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753a38e-98a7-405a-a6c5-cebebbe49db1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.671194 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1367/2705] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:37.676027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df17436-ff9c-4e54-b4fe-e078e460c061 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.686143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df17436-ff9c-4e54-b4fe-e078e460c061 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.691046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.691700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.691976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.692245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] PATH_INFO: `/v3/projects/f175d07458f6483a9e2b83d61e5293db/users/586fb3d9518341a3b543a72d05825209/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.694458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f175d07458f6483a9e2b83d61e5293db, user_id=586fb3d9518341a3b543a72d05825209, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.697089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.707521 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1339/2706] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:37 2022] PUT /identity/v3/projects/f175d07458f6483a9e2b83d61e5293db/users/586fb3d9518341a3b543a72d05825209/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:37.719325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd66116-599e-45bf-aca8-207203f095eb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.719792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd66116-599e-45bf-aca8-207203f095eb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.720070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd66116-599e-45bf-aca8-207203f095eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.748713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8bd66116-599e-45bf-aca8-207203f095eb None None] MFA Rules not processed for user `586fb3d9518341a3b543a72d05825209`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:37.750081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8bd66116-599e-45bf-aca8-207203f095eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:37.818784 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1368/2707] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:37.830170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:37.831991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:37.888683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17717446-bc00-409c-93cd-2d2bf557c944 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:37.892819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75ae4b-f6de-41be-a324-99d354a36d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:37.893405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75ae4b-f6de-41be-a324-99d354a36d10 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:37.893713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75ae4b-f6de-41be-a324-99d354a36d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:37.893984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75ae4b-f6de-41be-a324-99d354a36d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:37.902990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75ae4b-f6de-41be-a324-99d354a36d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:37.904413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75ae4b-f6de-41be-a324-99d354a36d10 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:37.914378 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1340/2708] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:37.982386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd66116-599e-45bf-aca8-207203f095eb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:38.016841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:38.045525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd66116-599e-45bf-aca8-207203f095eb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:38.050367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:38.050988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:38.051266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:38.051535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] PATH_INFO: `/v3/users/0ae40cbcf39d44aa976de8dcbf63dd32` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:38.052648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae40cbcf39d44aa976de8dcbf63dd32)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:38.054616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:38.099696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae40cbcf39d44aa976de8dcbf63dd32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.106788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae40cbcf39d44aa976de8dcbf63dd32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.107484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae40cbcf39d44aa976de8dcbf63dd32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.108088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae40cbcf39d44aa976de8dcbf63dd32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.116641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae40cbcf39d44aa976de8dcbf63dd32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.134832 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1369/2709] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:37 2022] DELETE /identity/v3/users/0ae40cbcf39d44aa976de8dcbf63dd32 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:38.139589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f445ae9-7205-4c29-bc6e-b1fd5f636daf None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:38.139894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f445ae9-7205-4c29-bc6e-b1fd5f636daf None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:38.140686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f445ae9-7205-4c29-bc6e-b1fd5f636daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:38.160124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8f445ae9-7205-4c29-bc6e-b1fd5f636daf None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:38.161323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8f445ae9-7205-4c29-bc6e-b1fd5f636daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:38.173687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:38.219968 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1341/2710] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:38.229467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:38.230932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:38.287512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bb0a2-8965-4c57-8fa7-05ce09fb3dae admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:38.291118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9cf2c-66f8-4676-a2cc-44fdc558535a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:38.291712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9cf2c-66f8-4676-a2cc-44fdc558535a service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:38.291987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9cf2c-66f8-4676-a2cc-44fdc558535a service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:38.292253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9cf2c-66f8-4676-a2cc-44fdc558535a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:38.301121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9cf2c-66f8-4676-a2cc-44fdc558535a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:38.302571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9cf2c-66f8-4676-a2cc-44fdc558535a service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:38.311750 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1370/2711] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:38.577926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f445ae9-7205-4c29-bc6e-b1fd5f636daf None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:38.579358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8f445ae9-7205-4c29-bc6e-b1fd5f636daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:38.591447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f445ae9-7205-4c29-bc6e-b1fd5f636daf None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:38.597091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:38.597759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:38.598055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:38.598311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] PATH_INFO: `/v3/projects/0b1339eb65394485942d6e5b3db3578b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:38.599486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1339eb65394485942d6e5b3db3578b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:38.601399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:38.651274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1339eb65394485942d6e5b3db3578b', 'request_id': 'req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.659468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1339eb65394485942d6e5b3db3578b', 'request_id': 'req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.665915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1339eb65394485942d6e5b3db3578b', 'request_id': 'req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.676671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] The token cache is being invalidate because project 0b1339eb65394485942d6e5b3db3578b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:38.676958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.677598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.679102 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1342/2712] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:38 2022] DELETE /identity/v3/projects/0b1339eb65394485942d6e5b3db3578b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:38.683146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9cf2c-66f8-4676-a2cc-44fdc558535a service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:38.692672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9cf2c-66f8-4676-a2cc-44fdc558535a service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:38.743772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:38.752446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:38.753083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:38.753556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:38.753919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] PATH_INFO: `/v3/users/586fb3d9518341a3b543a72d05825209` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:38.755264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=586fb3d9518341a3b543a72d05825209)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:38.757222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:38.802518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '586fb3d9518341a3b543a72d05825209'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.809791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '586fb3d9518341a3b543a72d05825209'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.810374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '586fb3d9518341a3b543a72d05825209'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.810981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '586fb3d9518341a3b543a72d05825209'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.818727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-061fd203-83b6-42ef-a878-36e7f496b9a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '586fb3d9518341a3b543a72d05825209'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:38.837497 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1371/2713] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:38 2022] DELETE /identity/v3/users/586fb3d9518341a3b543a72d05825209 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:39.192684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:39.194166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:39.210807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1a05d-e888-4c0d-9f1d-16126be1b2dd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:39.270949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:39.271586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.271864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.272136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] PATH_INFO: `/v3/projects/f175d07458f6483a9e2b83d61e5293db` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.273244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f175d07458f6483a9e2b83d61e5293db)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:39.275180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:39.318490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:39.321348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f175d07458f6483a9e2b83d61e5293db', 'request_id': 'req-f96afe5f-b510-4e51-8592-aad53925224c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:39.329388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f175d07458f6483a9e2b83d61e5293db', 'request_id': 'req-f96afe5f-b510-4e51-8592-aad53925224c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:39.335735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f175d07458f6483a9e2b83d61e5293db', 'request_id': 'req-f96afe5f-b510-4e51-8592-aad53925224c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:39.347214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] The token cache is being invalidate because project f175d07458f6483a9e2b83d61e5293db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:39.347500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:39.348129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f96afe5f-b510-4e51-8592-aad53925224c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:39.349595 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1343/2714] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:39 2022] DELETE /identity/v3/projects/f175d07458f6483a9e2b83d61e5293db => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:45:39.354557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b780-adcc-4014-ac77-e81937e61dd4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.354849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b780-adcc-4014-ac77-e81937e61dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.355129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b780-adcc-4014-ac77-e81937e61dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.375168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6435b780-adcc-4014-ac77-e81937e61dd4 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:39.376366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6435b780-adcc-4014-ac77-e81937e61dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:39.439325 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1372/2715] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:39.447890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4befbc-0d62-4dc1-9515-834630e23090 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.448180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4befbc-0d62-4dc1-9515-834630e23090 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.448458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4befbc-0d62-4dc1-9515-834630e23090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.471441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1d4befbc-0d62-4dc1-9515-834630e23090 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:39.472679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1d4befbc-0d62-4dc1-9515-834630e23090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:39.481385 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1344/2716] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:39.485990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435b780-adcc-4014-ac77-e81937e61dd4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:39.495618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435b780-adcc-4014-ac77-e81937e61dd4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:39.500240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:39.500838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.501113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.501379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.502047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:45:39.502345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:39.503943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:39.516463 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1373/2717] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:45:39.520910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4befbc-0d62-4dc1-9515-834630e23090 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:39.530492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4befbc-0d62-4dc1-9515-834630e23090 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:39.536468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30443e7c-2945-44a1-acfd-b534f2ce8d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:39.537075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30443e7c-2945-44a1-acfd-b534f2ce8d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.537355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30443e7c-2945-44a1-acfd-b534f2ce8d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.537651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30443e7c-2945-44a1-acfd-b534f2ce8d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.538298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30443e7c-2945-44a1-acfd-b534f2ce8d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:39.540198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30443e7c-2945-44a1-acfd-b534f2ce8d80 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:39.567857 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1345/2718] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:39 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:39.572246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:39.582533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f310a9c-ea77-4334-8bab-f699fe5a07b4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:39.644492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58b78d-28ef-4a0b-b477-5c488e965dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:39.645106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58b78d-28ef-4a0b-b477-5c488e965dbe admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.645391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58b78d-28ef-4a0b-b477-5c488e965dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.645690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58b78d-28ef-4a0b-b477-5c488e965dbe admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.646330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58b78d-28ef-4a0b-b477-5c488e965dbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:39.648300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58b78d-28ef-4a0b-b477-5c488e965dbe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:39.658694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:39.673659 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1374/2719] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:39 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:45:39.676880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30443e7c-2945-44a1-acfd-b534f2ce8d80 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:39.686852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30443e7c-2945-44a1-acfd-b534f2ce8d80 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:39.691627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40fcd-9825-4a59-ad32-5f38f032c93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:39.692167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40fcd-9825-4a59-ad32-5f38f032c93a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.692444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40fcd-9825-4a59-ad32-5f38f032c93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.692714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40fcd-9825-4a59-ad32-5f38f032c93a admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.693353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a40fcd-9825-4a59-ad32-5f38f032c93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:39.694964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a40fcd-9825-4a59-ad32-5f38f032c93a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:39.705386 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1346/2720] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:39.709482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58b78d-28ef-4a0b-b477-5c488e965dbe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:39.720386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58b78d-28ef-4a0b-b477-5c488e965dbe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:39.725229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdc1ec-eb9d-4022-914c-f11cd9def3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:39.725872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc1ec-eb9d-4022-914c-f11cd9def3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.726160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc1ec-eb9d-4022-914c-f11cd9def3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.726438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc1ec-eb9d-4022-914c-f11cd9def3a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.727137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdc1ec-eb9d-4022-914c-f11cd9def3a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:39.728762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdc1ec-eb9d-4022-914c-f11cd9def3a6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:39.740031 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1375/2721] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:39.744495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40fcd-9825-4a59-ad32-5f38f032c93a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:39.757184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40fcd-9825-4a59-ad32-5f38f032c93a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:39.761844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95fa6a-03f9-44fd-bc6c-15f7389d44ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:39.762523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95fa6a-03f9-44fd-bc6c-15f7389d44ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.762817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95fa6a-03f9-44fd-bc6c-15f7389d44ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.763087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95fa6a-03f9-44fd-bc6c-15f7389d44ad admin admin] PATH_INFO: `/v3/projects/c4e8e599a6594cadadcb453f2e998c24/users/c3d1c3172f34452782fc6507e96ff648/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.778742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95fa6a-03f9-44fd-bc6c-15f7389d44ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e8e599a6594cadadcb453f2e998c24, user_id=c3d1c3172f34452782fc6507e96ff648, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:39.782234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95fa6a-03f9-44fd-bc6c-15f7389d44ad admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:39.795226 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1347/2722] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:39 2022] PUT /identity/v3/projects/c4e8e599a6594cadadcb453f2e998c24/users/c3d1c3172f34452782fc6507e96ff648/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:39.805804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ba5a-5c62-42ca-832c-a3bc11f8b5c5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.806281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ba5a-5c62-42ca-832c-a3bc11f8b5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.806413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5ba5a-5c62-42ca-832c-a3bc11f8b5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.835656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7cc5ba5a-5c62-42ca-832c-a3bc11f8b5c5 None None] MFA Rules not processed for user `c3d1c3172f34452782fc6507e96ff648`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:39.836975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7cc5ba5a-5c62-42ca-832c-a3bc11f8b5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:39.903515 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1376/2723] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:39.908484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95fa6a-03f9-44fd-bc6c-15f7389d44ad admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:39.974205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95fa6a-03f9-44fd-bc6c-15f7389d44ad admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:39.979045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92cbae-b9ba-4d50-8763-f6b423602c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:39.979653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92cbae-b9ba-4d50-8763-f6b423602c90 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:39.979935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92cbae-b9ba-4d50-8763-f6b423602c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:39.980205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92cbae-b9ba-4d50-8763-f6b423602c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:39.980851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92cbae-b9ba-4d50-8763-f6b423602c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:39.983036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92cbae-b9ba-4d50-8763-f6b423602c90 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.012837 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1348/2724] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:39 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:40.018143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5ba5a-5c62-42ca-832c-a3bc11f8b5c5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.030871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5ba5a-5c62-42ca-832c-a3bc11f8b5c5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.097107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bfb9d-da5a-44aa-900d-328bc6e08ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.097782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bfb9d-da5a-44aa-900d-328bc6e08ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.098070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bfb9d-da5a-44aa-900d-328bc6e08ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.098346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bfb9d-da5a-44aa-900d-328bc6e08ddf admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.098984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bfb9d-da5a-44aa-900d-328bc6e08ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.100933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bfb9d-da5a-44aa-900d-328bc6e08ddf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.129737 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1377/2725] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:40 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:40.131055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92cbae-b9ba-4d50-8763-f6b423602c90 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.143775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92cbae-b9ba-4d50-8763-f6b423602c90 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.148499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd848e8-095c-4a7e-8064-b485b090ece0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.149103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd848e8-095c-4a7e-8064-b485b090ece0 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.149385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd848e8-095c-4a7e-8064-b485b090ece0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.149693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd848e8-095c-4a7e-8064-b485b090ece0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.150345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd848e8-095c-4a7e-8064-b485b090ece0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.151940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd848e8-095c-4a7e-8064-b485b090ece0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.164893 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1349/2726] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:40.168917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bfb9d-da5a-44aa-900d-328bc6e08ddf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.179433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bfb9d-da5a-44aa-900d-328bc6e08ddf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.184073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aad03e-eaac-4213-bc74-f601cf6ccc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.184700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aad03e-eaac-4213-bc74-f601cf6ccc17 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.184981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aad03e-eaac-4213-bc74-f601cf6ccc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.185254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aad03e-eaac-4213-bc74-f601cf6ccc17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.185963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aad03e-eaac-4213-bc74-f601cf6ccc17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.188222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aad03e-eaac-4213-bc74-f601cf6ccc17 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.198399 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1378/2727] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:40.203695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd848e8-095c-4a7e-8064-b485b090ece0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.215217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd848e8-095c-4a7e-8064-b485b090ece0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.220100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4e833-5f5b-4840-8c82-f895b7dc56af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.220793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4e833-5f5b-4840-8c82-f895b7dc56af admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.221076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4e833-5f5b-4840-8c82-f895b7dc56af admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.221381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4e833-5f5b-4840-8c82-f895b7dc56af admin admin] PATH_INFO: `/v3/projects/9ef9037f3df14a7e8b602b5394cdfbf3/users/109da50ee9ad4b60ab25483e8f7bc5f1/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.237297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4e833-5f5b-4840-8c82-f895b7dc56af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef9037f3df14a7e8b602b5394cdfbf3, user_id=109da50ee9ad4b60ab25483e8f7bc5f1, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.240047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4e833-5f5b-4840-8c82-f895b7dc56af admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.259540 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1350/2728] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:40 2022] PUT /identity/v3/projects/9ef9037f3df14a7e8b602b5394cdfbf3/users/109da50ee9ad4b60ab25483e8f7bc5f1/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:40.270343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793e7d6-c0dd-4185-913a-e879a1f7d784 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.270635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793e7d6-c0dd-4185-913a-e879a1f7d784 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.270913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793e7d6-c0dd-4185-913a-e879a1f7d784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.306065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a793e7d6-c0dd-4185-913a-e879a1f7d784 None None] MFA Rules not processed for user `109da50ee9ad4b60ab25483e8f7bc5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:40.307736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a793e7d6-c0dd-4185-913a-e879a1f7d784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:40.369511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:40.431380 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1379/2729] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:40.435780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4e833-5f5b-4840-8c82-f895b7dc56af admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.462134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:40.535539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4e833-5f5b-4840-8c82-f895b7dc56af admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.541160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f56c-a350-4e04-a002-faf4f16ca57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.542062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f56c-a350-4e04-a002-faf4f16ca57c admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.542635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f56c-a350-4e04-a002-faf4f16ca57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.543198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8f56c-a350-4e04-a002-faf4f16ca57c admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.544058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f56c-a350-4e04-a002-faf4f16ca57c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.546552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8f56c-a350-4e04-a002-faf4f16ca57c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.580472 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1351/2730] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:40 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:40.581703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793e7d6-c0dd-4185-913a-e879a1f7d784 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.592314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793e7d6-c0dd-4185-913a-e879a1f7d784 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.651833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058c19f-1211-4e8e-bbba-3f274d29cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.652453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c19f-1211-4e8e-bbba-3f274d29cb3b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.652736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c19f-1211-4e8e-bbba-3f274d29cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.653008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058c19f-1211-4e8e-bbba-3f274d29cb3b admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.653668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058c19f-1211-4e8e-bbba-3f274d29cb3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.655748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058c19f-1211-4e8e-bbba-3f274d29cb3b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.680869 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1380/2731] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:40 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:40.682055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f56c-a350-4e04-a002-faf4f16ca57c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.693215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8f56c-a350-4e04-a002-faf4f16ca57c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.698567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8cc0c-7b02-42ac-85ab-0ec70595fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.699175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8cc0c-7b02-42ac-85ab-0ec70595fb40 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.699462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8cc0c-7b02-42ac-85ab-0ec70595fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.699738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8cc0c-7b02-42ac-85ab-0ec70595fb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.700387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8cc0c-7b02-42ac-85ab-0ec70595fb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.702307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8cc0c-7b02-42ac-85ab-0ec70595fb40 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.713953 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1352/2732] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:40.717891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058c19f-1211-4e8e-bbba-3f274d29cb3b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.728219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058c19f-1211-4e8e-bbba-3f274d29cb3b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.733020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2923-1bb7-4fe1-9a63-0fe36eaec9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.733727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2923-1bb7-4fe1-9a63-0fe36eaec9d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.734025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2923-1bb7-4fe1-9a63-0fe36eaec9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.734291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e2923-1bb7-4fe1-9a63-0fe36eaec9d3 admin admin] PATH_INFO: `/v3/projects/875c41c4a97b4b10ae22c1ab9a25dcfa/users/b431838bf0c94c6ba29e97e0e8ed71c9/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.750348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e2923-1bb7-4fe1-9a63-0fe36eaec9d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=875c41c4a97b4b10ae22c1ab9a25dcfa, user_id=b431838bf0c94c6ba29e97e0e8ed71c9, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.753128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e2923-1bb7-4fe1-9a63-0fe36eaec9d3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.767746 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1381/2733] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:40 2022] PUT /identity/v3/projects/875c41c4a97b4b10ae22c1ab9a25dcfa/users/b431838bf0c94c6ba29e97e0e8ed71c9/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:40.771643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8cc0c-7b02-42ac-85ab-0ec70595fb40 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.781841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8cc0c-7b02-42ac-85ab-0ec70595fb40 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.842467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab7428-530f-457c-9f85-7ddc109a5dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.843603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab7428-530f-457c-9f85-7ddc109a5dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.843884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab7428-530f-457c-9f85-7ddc109a5dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.844341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab7428-530f-457c-9f85-7ddc109a5dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.845006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab7428-530f-457c-9f85-7ddc109a5dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.846636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab7428-530f-457c-9f85-7ddc109a5dc5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.861421 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1353/2734] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:40.862762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2923-1bb7-4fe1-9a63-0fe36eaec9d3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:40.873822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e2923-1bb7-4fe1-9a63-0fe36eaec9d3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:40.879578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:40.880242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.880520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.880790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] PATH_INFO: `/v3/projects/875c41c4a97b4b10ae22c1ab9a25dcfa/users/b431838bf0c94c6ba29e97e0e8ed71c9/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.882732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=875c41c4a97b4b10ae22c1ab9a25dcfa, user_id=b431838bf0c94c6ba29e97e0e8ed71c9, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:40.885567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:40.898415 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1382/2735] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:40 2022] PUT /identity/v3/projects/875c41c4a97b4b10ae22c1ab9a25dcfa/users/b431838bf0c94c6ba29e97e0e8ed71c9/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:45:40.910585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e5c81-bdde-4767-b81c-3f599123ae5c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:40.911610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e5c81-bdde-4767-b81c-3f599123ae5c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:40.911835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e5c81-bdde-4767-b81c-3f599123ae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:40.946783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4d3e5c81-bdde-4767-b81c-3f599123ae5c None None] MFA Rules not processed for user `b431838bf0c94c6ba29e97e0e8ed71c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:40.947994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4d3e5c81-bdde-4767-b81c-3f599123ae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:41.026243 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1354/2736] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:41.037846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:41.039678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:41.100603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb79a15-9552-49eb-86f4-02fcab4b41a3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:41.104205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:41.104803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:41.105082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:41.105353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:41.114158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:41.115616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:41.125027 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1383/2737] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:41.712268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e5c81-bdde-4767-b81c-3f599123ae5c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:41.713803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4d3e5c81-bdde-4767-b81c-3f599123ae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:41.726224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e5c81-bdde-4767-b81c-3f599123ae5c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:41.730038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41819d-2e4e-4019-a13d-7eeb43bd93a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:41.730650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41819d-2e4e-4019-a13d-7eeb43bd93a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:41.730930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41819d-2e4e-4019-a13d-7eeb43bd93a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:41.731206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41819d-2e4e-4019-a13d-7eeb43bd93a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:41.744454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:41.792133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41819d-2e4e-4019-a13d-7eeb43bd93a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:41.793612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41819d-2e4e-4019-a13d-7eeb43bd93a3 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:41.809724 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1355/2738] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:44.323536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:44.324919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:44.362326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:44.386719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38afdb-9b79-4fd6-9598-601279ea1492 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:44.390729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:44.391337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:44.391620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:44.391893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:44.393737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:44.464233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:44.465962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:44.477109 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1384/2739] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:45.912529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41819d-2e4e-4019-a13d-7eeb43bd93a3 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:45.922755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41819d-2e4e-4019-a13d-7eeb43bd93a3 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:45.926779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:45.927394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:45.927676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:45.927964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:45.997381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:45.999045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:46.009664 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1356/2740] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:48.000336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.001786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.053492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.055355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.074451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09b2eb-c5dc-49c4-b49d-8c58acff6f94 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.079246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.079908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.080185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.080653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] PATH_INFO: `/v3/users/8efb5824b0c74435b41f3d7911c20874` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.082444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8efb5824b0c74435b41f3d7911c20874)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.084554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.094348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc7f26-0d0d-4ef7-91b8-ebd30de26a7c service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.099624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.100242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.100494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.100761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.102698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.116037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.117740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.130050 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1385/2741] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:48.151439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8efb5824b0c74435b41f3d7911c20874'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.160668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8efb5824b0c74435b41f3d7911c20874'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.161442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8efb5824b0c74435b41f3d7911c20874'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.162189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8efb5824b0c74435b41f3d7911c20874'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.169565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8efb5824b0c74435b41f3d7911c20874'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.189337 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1357/2742] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:48 2022] DELETE /identity/v3/users/8efb5824b0c74435b41f3d7911c20874 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:48.289209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.301299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04061ce5-b000-4643-8be8-59bcd0c57505 service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.357970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.359521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.359918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.360199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] PATH_INFO: `/v3/users/c3d1c3172f34452782fc6507e96ff648` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.361357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3d1c3172f34452782fc6507e96ff648)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.363342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.381049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.381847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:48.383423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.400490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83899-4516-4114-bdf8-8ce06db24db6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.412151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.413054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.413336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.413806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] PATH_INFO: `/v3/projects/7258740d324b4be18675101826a78ca7` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.415195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7258740d324b4be18675101826a78ca7)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.417688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.424481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3d1c3172f34452782fc6507e96ff648'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.431886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3d1c3172f34452782fc6507e96ff648'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.432582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3d1c3172f34452782fc6507e96ff648'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.433209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3d1c3172f34452782fc6507e96ff648'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.441046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-500ac21a-bfe9-465e-9537-11a101e255d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3d1c3172f34452782fc6507e96ff648'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.441972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:48.461037 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1386/2743] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:48 2022] DELETE /identity/v3/users/c3d1c3172f34452782fc6507e96ff648 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:48.465304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bacd-1be2-4655-8421-49edc0ce9743 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.465604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bacd-1be2-4655-8421-49edc0ce9743 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.466019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3bacd-1be2-4655-8421-49edc0ce9743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.471790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7258740d324b4be18675101826a78ca7', 'request_id': 'req-f4451165-4376-4d36-9c43-b8116c41326c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.478734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7258740d324b4be18675101826a78ca7', 'request_id': 'req-f4451165-4376-4d36-9c43-b8116c41326c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.485200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7258740d324b4be18675101826a78ca7', 'request_id': 'req-f4451165-4376-4d36-9c43-b8116c41326c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.485890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3cb3bacd-1be2-4655-8421-49edc0ce9743 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:48.487173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3cb3bacd-1be2-4655-8421-49edc0ce9743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.496158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] The token cache is being invalidate because project 7258740d324b4be18675101826a78ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:48.496449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.497103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f4451165-4376-4d36-9c43-b8116c41326c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:48.498620 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1358/2744] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:48 2022] DELETE /identity/v3/projects/7258740d324b4be18675101826a78ca7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:48.503322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825b304-9192-47ac-b2ca-339615b2dcaa None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.503608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825b304-9192-47ac-b2ca-339615b2dcaa None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.503889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825b304-9192-47ac-b2ca-339615b2dcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.524128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e825b304-9192-47ac-b2ca-339615b2dcaa None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:48.525350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e825b304-9192-47ac-b2ca-339615b2dcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.547693 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1387/2745] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:45:48.557463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3bacd-1be2-4655-8421-49edc0ce9743 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.559019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3cb3bacd-1be2-4655-8421-49edc0ce9743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.585172 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1359/2746] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:45:48.593570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f025a6d-dcd2-43f5-97ab-97d28478d30a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.593943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f025a6d-dcd2-43f5-97ab-97d28478d30a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.594218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f025a6d-dcd2-43f5-97ab-97d28478d30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.613581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8f025a6d-dcd2-43f5-97ab-97d28478d30a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:48.614813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8f025a6d-dcd2-43f5-97ab-97d28478d30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.616943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3bacd-1be2-4655-8421-49edc0ce9743 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.620567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba322a2-ea69-4f74-9add-8b90d02f6644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.621199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba322a2-ea69-4f74-9add-8b90d02f6644 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.621482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba322a2-ea69-4f74-9add-8b90d02f6644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.621787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba322a2-ea69-4f74-9add-8b90d02f6644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.622723 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1360/2747] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:45:48.627070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f025a6d-dcd2-43f5-97ab-97d28478d30a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.631484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba322a2-ea69-4f74-9add-8b90d02f6644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.632935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba322a2-ea69-4f74-9add-8b90d02f6644 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.636909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f025a6d-dcd2-43f5-97ab-97d28478d30a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.642654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.643269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.643486 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1388/2748] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:45:48.643920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.644202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.644921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:45:48.645225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.646847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.659606 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1361/2749] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:45:48.664097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba322a2-ea69-4f74-9add-8b90d02f6644 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.674477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba322a2-ea69-4f74-9add-8b90d02f6644 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.679092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbf01a-9e1b-444a-9ba1-88c5fcb002cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.679761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbf01a-9e1b-444a-9ba1-88c5fcb002cf admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.680141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbf01a-9e1b-444a-9ba1-88c5fcb002cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.680488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbf01a-9e1b-444a-9ba1-88c5fcb002cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.681125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbf01a-9e1b-444a-9ba1-88c5fcb002cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.683088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbf01a-9e1b-444a-9ba1-88c5fcb002cf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.711621 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1389/2750] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:48 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:48.716366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.727348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554d7d4-8906-41ee-adce-4ffd0f556def admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.787116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75997cfc-338c-4528-af03-a6b9bba7087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.787730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75997cfc-338c-4528-af03-a6b9bba7087c admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.788012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75997cfc-338c-4528-af03-a6b9bba7087c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.788362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75997cfc-338c-4528-af03-a6b9bba7087c admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.788909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75997cfc-338c-4528-af03-a6b9bba7087c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.790902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75997cfc-338c-4528-af03-a6b9bba7087c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.817660 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1362/2751] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:48 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:48.819026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbf01a-9e1b-444a-9ba1-88c5fcb002cf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.829659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbf01a-9e1b-444a-9ba1-88c5fcb002cf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.835236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71590c54-6e1a-4dbc-87b3-cfd4f59ac0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.836267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71590c54-6e1a-4dbc-87b3-cfd4f59ac0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.836769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71590c54-6e1a-4dbc-87b3-cfd4f59ac0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.837241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71590c54-6e1a-4dbc-87b3-cfd4f59ac0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.838466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71590c54-6e1a-4dbc-87b3-cfd4f59ac0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.840861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71590c54-6e1a-4dbc-87b3-cfd4f59ac0c0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.852521 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1390/2752] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:48.856343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75997cfc-338c-4528-af03-a6b9bba7087c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.866449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75997cfc-338c-4528-af03-a6b9bba7087c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.871139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.871743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.872023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.872298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.872946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.875126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.885931 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1363/2753] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:48.891147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71590c54-6e1a-4dbc-87b3-cfd4f59ac0c0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.897812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:48.903121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71590c54-6e1a-4dbc-87b3-cfd4f59ac0c0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.909728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b903a-7e6c-49c7-9aed-a2527134e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.910479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b903a-7e6c-49c7-9aed-a2527134e496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.910770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b903a-7e6c-49c7-9aed-a2527134e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.911350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b903a-7e6c-49c7-9aed-a2527134e496 admin admin] PATH_INFO: `/v3/projects/319bd365127d481aac9f127196321ecf/users/aec22e5f030d43fe8de90d17fe8225dc/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.915870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:48.917977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.932642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455a351-f7fc-49e5-b1c2-f1378841beb7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:48.936374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b903a-7e6c-49c7-9aed-a2527134e496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=319bd365127d481aac9f127196321ecf, user_id=aec22e5f030d43fe8de90d17fe8225dc, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.938168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:48.938755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.939013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.939282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] PATH_INFO: `/v3/projects/c4e8e599a6594cadadcb453f2e998c24` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.939634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b903a-7e6c-49c7-9aed-a2527134e496 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.940502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e8e599a6594cadadcb453f2e998c24)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:48.942488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:48.944505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:48.958473 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1391/2754] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:48 2022] PUT /identity/v3/projects/319bd365127d481aac9f127196321ecf/users/aec22e5f030d43fe8de90d17fe8225dc/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:48.963271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf7982-d9e3-47cd-8777-08e7b4deeceb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:48.963627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf7982-d9e3-47cd-8777-08e7b4deeceb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:48.963973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf7982-d9e3-47cd-8777-08e7b4deeceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:48.982729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a4cf7982-d9e3-47cd-8777-08e7b4deeceb None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:48.984055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a4cf7982-d9e3-47cd-8777-08e7b4deeceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:48.994720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e8e599a6594cadadcb453f2e998c24', 'request_id': 'req-847632b0-810a-4b5a-bb2d-38736ba46418'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.003124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e8e599a6594cadadcb453f2e998c24', 'request_id': 'req-847632b0-810a-4b5a-bb2d-38736ba46418'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.009421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e8e599a6594cadadcb453f2e998c24', 'request_id': 'req-847632b0-810a-4b5a-bb2d-38736ba46418'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.024928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] The token cache is being invalidate because project c4e8e599a6594cadadcb453f2e998c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:49.025290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.026079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.028512 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1364/2755] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:48 2022] DELETE /identity/v3/projects/c4e8e599a6594cadadcb453f2e998c24 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:45:49.032369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:49.033835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:49.058093 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1392/2756] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:45:49.068514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf7982-d9e3-47cd-8777-08e7b4deeceb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:49.070422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a4cf7982-d9e3-47cd-8777-08e7b4deeceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:49.106702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847632b0-810a-4b5a-bb2d-38736ba46418 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:49.113215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:49.113215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:49.113215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:49.113215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] PATH_INFO: `/v3/users/109da50ee9ad4b60ab25483e8f7bc5f1` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:49.114556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=109da50ee9ad4b60ab25483e8f7bc5f1)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:49.116552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:49.132604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf7982-d9e3-47cd-8777-08e7b4deeceb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:49.143435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94c28-510b-427b-91f6-3ac4acad381e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:49.144087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94c28-510b-427b-91f6-3ac4acad381e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:49.144364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94c28-510b-427b-91f6-3ac4acad381e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:49.144677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94c28-510b-427b-91f6-3ac4acad381e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:49.155519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a94c28-510b-427b-91f6-3ac4acad381e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:49.157316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a94c28-510b-427b-91f6-3ac4acad381e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:49.167071 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1393/2757] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:49.170209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109da50ee9ad4b60ab25483e8f7bc5f1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.180881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109da50ee9ad4b60ab25483e8f7bc5f1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.181951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109da50ee9ad4b60ab25483e8f7bc5f1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.182902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109da50ee9ad4b60ab25483e8f7bc5f1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.191375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109da50ee9ad4b60ab25483e8f7bc5f1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.211277 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1365/2758] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:49 2022] DELETE /identity/v3/users/109da50ee9ad4b60ab25483e8f7bc5f1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:49.494218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94c28-510b-427b-91f6-3ac4acad381e service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:49.508044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94c28-510b-427b-91f6-3ac4acad381e service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:49.578732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:49.578966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:49.579214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:49.579487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] PATH_INFO: `/v3/projects/9ef9037f3df14a7e8b602b5394cdfbf3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:49.580624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ef9037f3df14a7e8b602b5394cdfbf3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:49.582803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:49.612930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:49.637649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ef9037f3df14a7e8b602b5394cdfbf3', 'request_id': 'req-118835f9-f73d-4a1d-8c83-a88f341dcce3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.646901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ef9037f3df14a7e8b602b5394cdfbf3', 'request_id': 'req-118835f9-f73d-4a1d-8c83-a88f341dcce3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.653646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ef9037f3df14a7e8b602b5394cdfbf3', 'request_id': 'req-118835f9-f73d-4a1d-8c83-a88f341dcce3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.665326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] The token cache is being invalidate because project 9ef9037f3df14a7e8b602b5394cdfbf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:49.665613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.666291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.667836 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1394/2759] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:49 2022] DELETE /identity/v3/projects/9ef9037f3df14a7e8b602b5394cdfbf3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:49.673588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:49.675185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:49.688852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ac604-163e-47a5-bcff-e7dbeec44ec3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:49.693972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:49.748822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:49.749813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:49.749968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:49.750185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] PATH_INFO: `/v3/users/b431838bf0c94c6ba29e97e0e8ed71c9` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:49.751317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b431838bf0c94c6ba29e97e0e8ed71c9)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:49.753665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:49.805188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b431838bf0c94c6ba29e97e0e8ed71c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.814372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b431838bf0c94c6ba29e97e0e8ed71c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.815426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b431838bf0c94c6ba29e97e0e8ed71c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.816243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b431838bf0c94c6ba29e97e0e8ed71c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.829260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa47d009-bf63-496e-8dfd-5fea69b099e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b431838bf0c94c6ba29e97e0e8ed71c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:49.850881 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1366/2760] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:49 2022] DELETE /identity/v3/users/b431838bf0c94c6ba29e97e0e8ed71c9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:50.178176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:50.180011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:50.199861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118835f9-f73d-4a1d-8c83-a88f341dcce3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:50.275895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:50.276534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.276811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.277078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] PATH_INFO: `/v3/projects/875c41c4a97b4b10ae22c1ab9a25dcfa` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.278212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=875c41c4a97b4b10ae22c1ab9a25dcfa)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:50.280121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:50.341041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '875c41c4a97b4b10ae22c1ab9a25dcfa', 'request_id': 'req-11adfd71-ac48-4e10-9031-d62d713c7f41'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:50.351734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '875c41c4a97b4b10ae22c1ab9a25dcfa', 'request_id': 'req-11adfd71-ac48-4e10-9031-d62d713c7f41'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:50.359166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '875c41c4a97b4b10ae22c1ab9a25dcfa', 'request_id': 'req-11adfd71-ac48-4e10-9031-d62d713c7f41'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:50.370398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] The token cache is being invalidate because project 875c41c4a97b4b10ae22c1ab9a25dcfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:50.370689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:50.371356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-11adfd71-ac48-4e10-9031-d62d713c7f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:50.373012 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1395/2761] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:50 2022] DELETE /identity/v3/projects/875c41c4a97b4b10ae22c1ab9a25dcfa => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:50.379625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87199cec-d0b9-4f67-8092-24147aefc0c3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.380129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87199cec-d0b9-4f67-8092-24147aefc0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.380349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87199cec-d0b9-4f67-8092-24147aefc0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.415800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-87199cec-d0b9-4f67-8092-24147aefc0c3 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:50.416342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-87199cec-d0b9-4f67-8092-24147aefc0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:50.493926 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1367/2762] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:50.505057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9347b8-6d0e-4b64-b3d3-1fa36aeae1a2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.505375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9347b8-6d0e-4b64-b3d3-1fa36aeae1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.505598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9347b8-6d0e-4b64-b3d3-1fa36aeae1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.530046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-af9347b8-6d0e-4b64-b3d3-1fa36aeae1a2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:50.531814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-af9347b8-6d0e-4b64-b3d3-1fa36aeae1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:50.540698 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1396/2763] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:50.550382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87199cec-d0b9-4f67-8092-24147aefc0c3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:50.561349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87199cec-d0b9-4f67-8092-24147aefc0c3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:50.568869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:50.569911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.570382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.570869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.571738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:45:50.572216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:50.574037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:50.608992 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1368/2764] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:45:50.613921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9347b8-6d0e-4b64-b3d3-1fa36aeae1a2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:50.624767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9347b8-6d0e-4b64-b3d3-1fa36aeae1a2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:50.629788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4bb4b-c80c-4f72-835e-239bf10e2047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:50.630355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4bb4b-c80c-4f72-835e-239bf10e2047 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.630640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4bb4b-c80c-4f72-835e-239bf10e2047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.630933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4bb4b-c80c-4f72-835e-239bf10e2047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.631554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4bb4b-c80c-4f72-835e-239bf10e2047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:50.633497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4bb4b-c80c-4f72-835e-239bf10e2047 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:50.666094 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1397/2765] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:50 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:50.670365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:50.681830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db207537-924c-420f-8ba1-f51d1b5c703b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:50.695412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:50.763037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4107ef-4695-42e0-81e8-068088a33ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:50.763631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4107ef-4695-42e0-81e8-068088a33ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.763907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4107ef-4695-42e0-81e8-068088a33ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.764487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4107ef-4695-42e0-81e8-068088a33ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.765207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4107ef-4695-42e0-81e8-068088a33ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:50.767369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4107ef-4695-42e0-81e8-068088a33ca9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:50.797493 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1369/2766] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:50 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:50.801853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4bb4b-c80c-4f72-835e-239bf10e2047 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:50.813851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4bb4b-c80c-4f72-835e-239bf10e2047 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:50.819669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2a590-b358-4431-af42-4a4caa64ede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:50.820422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a590-b358-4431-af42-4a4caa64ede7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.820781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a590-b358-4431-af42-4a4caa64ede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.822484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a590-b358-4431-af42-4a4caa64ede7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.822484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2a590-b358-4431-af42-4a4caa64ede7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:50.823884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2a590-b358-4431-af42-4a4caa64ede7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:50.834888 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1398/2767] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:45:50.839447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4107ef-4695-42e0-81e8-068088a33ca9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:50.852033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4107ef-4695-42e0-81e8-068088a33ca9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:50.857649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8f4ed-f1d2-43ec-91ed-32c7a1d18e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:50.858321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f4ed-f1d2-43ec-91ed-32c7a1d18e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.859207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f4ed-f1d2-43ec-91ed-32c7a1d18e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.859475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f4ed-f1d2-43ec-91ed-32c7a1d18e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.860491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8f4ed-f1d2-43ec-91ed-32c7a1d18e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:50.862223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8f4ed-f1d2-43ec-91ed-32c7a1d18e41 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:50.877094 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1370/2768] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:50.878403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2a590-b358-4431-af42-4a4caa64ede7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:50.891009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2a590-b358-4431-af42-4a4caa64ede7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:50.892084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:50.903724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc418f2-e8d5-40a8-9d49-b9e4f37a9219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:50.904924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc418f2-e8d5-40a8-9d49-b9e4f37a9219 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.906513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc418f2-e8d5-40a8-9d49-b9e4f37a9219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.906745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc418f2-e8d5-40a8-9d49-b9e4f37a9219 admin admin] PATH_INFO: `/v3/projects/55c0262b2e2f4e7bbc3c4ba6fc91597b/users/3909be4fde9542c3836556aee3b31dd0/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.930453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc418f2-e8d5-40a8-9d49-b9e4f37a9219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c0262b2e2f4e7bbc3c4ba6fc91597b, user_id=3909be4fde9542c3836556aee3b31dd0, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:50.933230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc418f2-e8d5-40a8-9d49-b9e4f37a9219 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:50.947134 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1399/2769] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:50 2022] PUT /identity/v3/projects/55c0262b2e2f4e7bbc3c4ba6fc91597b/users/3909be4fde9542c3836556aee3b31dd0/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:50.959164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77077be-1dd5-4ba1-b521-1bb7dc81c704 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:50.959444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77077be-1dd5-4ba1-b521-1bb7dc81c704 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:50.959715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77077be-1dd5-4ba1-b521-1bb7dc81c704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:50.987448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a77077be-1dd5-4ba1-b521-1bb7dc81c704 None None] MFA Rules not processed for user `3909be4fde9542c3836556aee3b31dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:50.988576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a77077be-1dd5-4ba1-b521-1bb7dc81c704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:51.065336 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1371/2770] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:51.069941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc418f2-e8d5-40a8-9d49-b9e4f37a9219 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:51.151378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc418f2-e8d5-40a8-9d49-b9e4f37a9219 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:51.157207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6b61e-f4cd-4130-8874-3ab4a195b951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:51.158079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b61e-f4cd-4130-8874-3ab4a195b951 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:51.158783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b61e-f4cd-4130-8874-3ab4a195b951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:51.159133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b61e-f4cd-4130-8874-3ab4a195b951 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:51.160097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6b61e-f4cd-4130-8874-3ab4a195b951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:51.162536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6b61e-f4cd-4130-8874-3ab4a195b951 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:51.199786 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1400/2771] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:51 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:51.200963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77077be-1dd5-4ba1-b521-1bb7dc81c704 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:51.218285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77077be-1dd5-4ba1-b521-1bb7dc81c704 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:51.300859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795cfb4-3038-4466-adfb-2503d70ae9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:51.301732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795cfb4-3038-4466-adfb-2503d70ae9dc admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:51.302337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795cfb4-3038-4466-adfb-2503d70ae9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:51.302947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795cfb4-3038-4466-adfb-2503d70ae9dc admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:51.304150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795cfb4-3038-4466-adfb-2503d70ae9dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:51.307012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795cfb4-3038-4466-adfb-2503d70ae9dc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:51.337019 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1372/2772] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:51 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:51.340971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6b61e-f4cd-4130-8874-3ab4a195b951 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:51.353864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6b61e-f4cd-4130-8874-3ab4a195b951 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:51.363642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210f5a9-7d73-49e5-a57f-5bec32fb6d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:51.364606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210f5a9-7d73-49e5-a57f-5bec32fb6d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:51.364991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210f5a9-7d73-49e5-a57f-5bec32fb6d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:51.365363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210f5a9-7d73-49e5-a57f-5bec32fb6d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:51.366242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210f5a9-7d73-49e5-a57f-5bec32fb6d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:51.368534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210f5a9-7d73-49e5-a57f-5bec32fb6d85 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:51.381287 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1401/2773] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:51.385608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795cfb4-3038-4466-adfb-2503d70ae9dc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:51.398254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795cfb4-3038-4466-adfb-2503d70ae9dc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:51.403104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df680ea1-7e9c-47b6-9f84-78b35ed17cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:51.404113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df680ea1-7e9c-47b6-9f84-78b35ed17cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:51.404113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df680ea1-7e9c-47b6-9f84-78b35ed17cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:51.404326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df680ea1-7e9c-47b6-9f84-78b35ed17cab admin admin] PATH_INFO: `/v3/projects/4293b25c9155482badd7261361431b09/users/4236fabb2854443ca7ae1183052963c8/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:51.419950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:51.429200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df680ea1-7e9c-47b6-9f84-78b35ed17cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4293b25c9155482badd7261361431b09, user_id=4236fabb2854443ca7ae1183052963c8, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:51.433790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df680ea1-7e9c-47b6-9f84-78b35ed17cab admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:51.459891 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1373/2774] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:51 2022] PUT /identity/v3/projects/4293b25c9155482badd7261361431b09/users/4236fabb2854443ca7ae1183052963c8/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:45:51.463816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210f5a9-7d73-49e5-a57f-5bec32fb6d85 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:51.476135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210f5a9-7d73-49e5-a57f-5bec32fb6d85 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:51.566006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40a1f1-f21c-43e4-aabf-7cdfbdff3aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:51.567421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a1f1-f21c-43e4-aabf-7cdfbdff3aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:51.567866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a1f1-f21c-43e4-aabf-7cdfbdff3aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:51.569072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a1f1-f21c-43e4-aabf-7cdfbdff3aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:51.570023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40a1f1-f21c-43e4-aabf-7cdfbdff3aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:51.573021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40a1f1-f21c-43e4-aabf-7cdfbdff3aa9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:51.590672 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1402/2775] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:51.594836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df680ea1-7e9c-47b6-9f84-78b35ed17cab admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:51.605711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df680ea1-7e9c-47b6-9f84-78b35ed17cab admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:51.611525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:51.612197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:51.612477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:51.612746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] PATH_INFO: `/v3/projects/4293b25c9155482badd7261361431b09/users/4236fabb2854443ca7ae1183052963c8/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:51.614900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4293b25c9155482badd7261361431b09, user_id=4236fabb2854443ca7ae1183052963c8, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:51.618032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:51.638152 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1374/2776] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:51 2022] PUT /identity/v3/projects/4293b25c9155482badd7261361431b09/users/4236fabb2854443ca7ae1183052963c8/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:51.643493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c036d95-5067-4e35-b48e-077e5dcfb9b0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:51.643782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c036d95-5067-4e35-b48e-077e5dcfb9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:51.644081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c036d95-5067-4e35-b48e-077e5dcfb9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:51.682365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7c036d95-5067-4e35-b48e-077e5dcfb9b0 None None] MFA Rules not processed for user `4236fabb2854443ca7ae1183052963c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:51.683733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7c036d95-5067-4e35-b48e-077e5dcfb9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:51.727523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:51.770342 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1403/2777] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:51.780127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:51.783347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:51.853620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ef1cb-fa0d-4e9b-a29d-df3044178a70 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:51.859478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde02e9-8de1-45dd-8480-2a8b444a64aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:51.860685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde02e9-8de1-45dd-8480-2a8b444a64aa service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:51.860685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde02e9-8de1-45dd-8480-2a8b444a64aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:51.860919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde02e9-8de1-45dd-8480-2a8b444a64aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:51.873174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde02e9-8de1-45dd-8480-2a8b444a64aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:51.875449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde02e9-8de1-45dd-8480-2a8b444a64aa service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:51.885987 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1375/2778] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:52.621064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c036d95-5067-4e35-b48e-077e5dcfb9b0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:52.696363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c036d95-5067-4e35-b48e-077e5dcfb9b0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:52.702399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:52.703035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:52.703311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:52.703577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] PATH_INFO: `/v3/users/3909be4fde9542c3836556aee3b31dd0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:52.704744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3909be4fde9542c3836556aee3b31dd0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:52.706742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:52.765823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3909be4fde9542c3836556aee3b31dd0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:52.774322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3909be4fde9542c3836556aee3b31dd0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:52.775063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3909be4fde9542c3836556aee3b31dd0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:52.775669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3909be4fde9542c3836556aee3b31dd0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:52.784470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3909be4fde9542c3836556aee3b31dd0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:52.808718 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1404/2779] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:52 2022] DELETE /identity/v3/users/3909be4fde9542c3836556aee3b31dd0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:52.810377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676e3f-cc96-430d-ad57-751ef8885922 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:52.810837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676e3f-cc96-430d-ad57-751ef8885922 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:52.810932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15676e3f-cc96-430d-ad57-751ef8885922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:52.831089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-15676e3f-cc96-430d-ad57-751ef8885922 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:52.832584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-15676e3f-cc96-430d-ad57-751ef8885922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:52.907343 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1376/2780] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:52.918286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:52.919623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:52.996806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ab6e5-922d-47ee-8f25-5e6cf7676b67 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:53.005855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:53.006961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:53.006961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:53.007151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:53.017912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:53.022915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:53.032313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:53.050425 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1405/2781] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:53.366717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d4b12-fa85-4bd0-9c87-6f5678f10062 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:53.367198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d4b12-fa85-4bd0-9c87-6f5678f10062 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:53.367491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d4b12-fa85-4bd0-9c87-6f5678f10062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:53.410159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d96d4b12-fa85-4bd0-9c87-6f5678f10062 None None] MFA Rules not processed for user `aec22e5f030d43fe8de90d17fe8225dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:53.411415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d96d4b12-fa85-4bd0-9c87-6f5678f10062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:53.493866 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1377/2782] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:53.508827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:53.510315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:53.518561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d4b12-fa85-4bd0-9c87-6f5678f10062 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:53.521859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d96d4b12-fa85-4bd0-9c87-6f5678f10062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:53.545953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d4b12-fa85-4bd0-9c87-6f5678f10062 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:53.563302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:53.564236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:53.564879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:53.566003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] PATH_INFO: `/v3/projects/55c0262b2e2f4e7bbc3c4ba6fc91597b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:53.571207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c0262b2e2f4e7bbc3c4ba6fc91597b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:53.574850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:53.578174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:53.638522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c0262b2e2f4e7bbc3c4ba6fc91597b', 'request_id': 'req-a3017896-bbc5-4370-89f2-6eba89d2a0ef'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.639457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488772-d347-42d0-9ae9-55d4d7eb2623 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:53.648542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c0262b2e2f4e7bbc3c4ba6fc91597b', 'request_id': 'req-a3017896-bbc5-4370-89f2-6eba89d2a0ef'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.657000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c0262b2e2f4e7bbc3c4ba6fc91597b', 'request_id': 'req-a3017896-bbc5-4370-89f2-6eba89d2a0ef'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.670307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] The token cache is being invalidate because project 55c0262b2e2f4e7bbc3c4ba6fc91597b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:53.670748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.671793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.673591 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1378/2783] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:53 2022] DELETE /identity/v3/projects/55c0262b2e2f4e7bbc3c4ba6fc91597b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:53.677401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:53.678879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:53.722789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:53.723409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:53.723693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:53.723965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:53.725908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:53.752864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3017896-bbc5-4370-89f2-6eba89d2a0ef admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:53.757750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:53.758391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:53.758670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:53.758943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] PATH_INFO: `/v3/users/4236fabb2854443ca7ae1183052963c8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:53.760098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4236fabb2854443ca7ae1183052963c8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:53.762072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:53.796796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:53.798476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:53.809646 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1406/2784] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:53.819596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4236fabb2854443ca7ae1183052963c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.829852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4236fabb2854443ca7ae1183052963c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.830993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4236fabb2854443ca7ae1183052963c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.831598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4236fabb2854443ca7ae1183052963c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.840544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bd384721-62d1-4d7d-8508-b0dc9d8323f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4236fabb2854443ca7ae1183052963c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:53.877682 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1379/2785] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:53 2022] DELETE /identity/v3/users/4236fabb2854443ca7ae1183052963c8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:45:54.227701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:54.240201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202d833-232f-4a32-8410-5b79d5c31501 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:54.291231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:54.323301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:54.323960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.324246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.324520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] PATH_INFO: `/v3/projects/4293b25c9155482badd7261361431b09` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.326148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4293b25c9155482badd7261361431b09)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:54.327781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:54.397434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4293b25c9155482badd7261361431b09', 'request_id': 'req-6011af54-943b-40a7-a3ac-aebfc0085473'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:54.410437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4293b25c9155482badd7261361431b09', 'request_id': 'req-6011af54-943b-40a7-a3ac-aebfc0085473'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:54.419155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4293b25c9155482badd7261361431b09', 'request_id': 'req-6011af54-943b-40a7-a3ac-aebfc0085473'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:54.434889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] The token cache is being invalidate because project 4293b25c9155482badd7261361431b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:54.435239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:54.435939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6011af54-943b-40a7-a3ac-aebfc0085473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:54.438826 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1407/2786] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:54 2022] DELETE /identity/v3/projects/4293b25c9155482badd7261361431b09 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:54.443708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cf599-0d73-49e0-8ca5-4e886a8d03bd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.444004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cf599-0d73-49e0-8ca5-4e886a8d03bd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.444282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cf599-0d73-49e0-8ca5-4e886a8d03bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.471257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7a3cf599-0d73-49e0-8ca5-4e886a8d03bd None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:54.472558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7a3cf599-0d73-49e0-8ca5-4e886a8d03bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:54.478421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:54.550869 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1380/2787] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:54.558846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1c5f4-4c92-4a48-ad7b-a599e09f9c06 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.559133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1c5f4-4c92-4a48-ad7b-a599e09f9c06 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.559405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1c5f4-4c92-4a48-ad7b-a599e09f9c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.585442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-70c1c5f4-4c92-4a48-ad7b-a599e09f9c06 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:54.587005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-70c1c5f4-4c92-4a48-ad7b-a599e09f9c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:54.598307 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1408/2788] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:54.602321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cf599-0d73-49e0-8ca5-4e886a8d03bd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:54.616587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cf599-0d73-49e0-8ca5-4e886a8d03bd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:54.621485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:54.622132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.622413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.622684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.623340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:45:54.623646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:54.625665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:54.641068 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1381/2789] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:45:54.645603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1c5f4-4c92-4a48-ad7b-a599e09f9c06 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:54.659656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1c5f4-4c92-4a48-ad7b-a599e09f9c06 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:54.665070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b55e6-042f-41eb-a24e-295bad0fab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:54.665746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b55e6-042f-41eb-a24e-295bad0fab3a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.666031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b55e6-042f-41eb-a24e-295bad0fab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.666307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b55e6-042f-41eb-a24e-295bad0fab3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.666965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b55e6-042f-41eb-a24e-295bad0fab3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:54.668902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b55e6-042f-41eb-a24e-295bad0fab3a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:54.699460 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1409/2790] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:54 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:54.704272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:54.714889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66bd32-4319-436d-90f2-8ede85815ef3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:54.778537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde2b19-08b3-423e-bed8-7b3ae5766646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:54.779192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde2b19-08b3-423e-bed8-7b3ae5766646 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.779479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde2b19-08b3-423e-bed8-7b3ae5766646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.779777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde2b19-08b3-423e-bed8-7b3ae5766646 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.781063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde2b19-08b3-423e-bed8-7b3ae5766646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:54.783055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde2b19-08b3-423e-bed8-7b3ae5766646 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:54.808374 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1382/2791] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:54 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:45:54.812918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b55e6-042f-41eb-a24e-295bad0fab3a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:54.829701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b55e6-042f-41eb-a24e-295bad0fab3a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:54.836024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba37be-7662-4555-826d-23c8fd576eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:54.838018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba37be-7662-4555-826d-23c8fd576eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.838881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba37be-7662-4555-826d-23c8fd576eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.838881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba37be-7662-4555-826d-23c8fd576eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.839629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba37be-7662-4555-826d-23c8fd576eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:54.841349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba37be-7662-4555-826d-23c8fd576eaa admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:54.860099 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1410/2792] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:45:54.864034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde2b19-08b3-423e-bed8-7b3ae5766646 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:54.874506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde2b19-08b3-423e-bed8-7b3ae5766646 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:54.884491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adb94e-4fd8-487c-9899-76f6befb20dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:54.885129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adb94e-4fd8-487c-9899-76f6befb20dd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.885415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adb94e-4fd8-487c-9899-76f6befb20dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.885792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adb94e-4fd8-487c-9899-76f6befb20dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.886459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adb94e-4fd8-487c-9899-76f6befb20dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:54.888105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adb94e-4fd8-487c-9899-76f6befb20dd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:54.904745 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1383/2793] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:45:54.909718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba37be-7662-4555-826d-23c8fd576eaa admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:54.920044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba37be-7662-4555-826d-23c8fd576eaa admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:54.925706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:54.926370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.926680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.926959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] PATH_INFO: `/v3/projects/18370bede7df4cf9b89802805c3885b5/users/449032b6aee742969755d512d82d354b/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:54.941913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18370bede7df4cf9b89802805c3885b5, user_id=449032b6aee742969755d512d82d354b, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:54.944635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:54.961531 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1411/2794] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:54 2022] PUT /identity/v3/projects/18370bede7df4cf9b89802805c3885b5/users/449032b6aee742969755d512d82d354b/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:54.972478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65a98f-435f-4cb8-9a17-866851fbc8a8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:54.972768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65a98f-435f-4cb8-9a17-866851fbc8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:54.973044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65a98f-435f-4cb8-9a17-866851fbc8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:55.004232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8a65a98f-435f-4cb8-9a17-866851fbc8a8 None None] MFA Rules not processed for user `449032b6aee742969755d512d82d354b`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:55.005467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8a65a98f-435f-4cb8-9a17-866851fbc8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:55.099092 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1384/2795] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:55.112575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:55.114645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:55.187103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc161131-150f-4b3c-b8d1-4db329bb9f7c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:55.191806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b3eb8-415e-4a08-915c-382733d0f38c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:55.192538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3eb8-415e-4a08-915c-382733d0f38c service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:55.192819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3eb8-415e-4a08-915c-382733d0f38c service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:55.193090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b3eb8-415e-4a08-915c-382733d0f38c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:55.205465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b3eb8-415e-4a08-915c-382733d0f38c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:55.207018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b3eb8-415e-4a08-915c-382733d0f38c service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:55.213740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:55.225384 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1412/2796] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:55.252130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65a98f-435f-4cb8-9a17-866851fbc8a8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:55.263186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:55.308449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b78f47-5b36-4d16-99f1-83797f887aa5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:55.308835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b78f47-5b36-4d16-99f1-83797f887aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:55.309180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b78f47-5b36-4d16-99f1-83797f887aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:55.336545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f0b78f47-5b36-4d16-99f1-83797f887aa5 None None] MFA Rules not processed for user `aec22e5f030d43fe8de90d17fe8225dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:55.338110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f0b78f47-5b36-4d16-99f1-83797f887aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:55.370251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65a98f-435f-4cb8-9a17-866851fbc8a8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:55.376276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:55.377681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:55.378229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:55.378722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] PATH_INFO: `/v3/users/449032b6aee742969755d512d82d354b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:55.381052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=449032b6aee742969755d512d82d354b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:55.385430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:55.424329 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1413/2797] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:55.445266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '449032b6aee742969755d512d82d354b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:55.460669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '449032b6aee742969755d512d82d354b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:55.461499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '449032b6aee742969755d512d82d354b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:55.462406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '449032b6aee742969755d512d82d354b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:55.476431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '449032b6aee742969755d512d82d354b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:55.503623 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1385/2798] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:55 2022] DELETE /identity/v3/users/449032b6aee742969755d512d82d354b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:55.508583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb49e4a-5a5f-4953-abde-c1950ac59cc8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:55.509021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb49e4a-5a5f-4953-abde-c1950ac59cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:55.509375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb49e4a-5a5f-4953-abde-c1950ac59cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:55.539078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1fb49e4a-5a5f-4953-abde-c1950ac59cc8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:55.540426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1fb49e4a-5a5f-4953-abde-c1950ac59cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:55.617951 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1414/2799] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:55.628059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:55.629564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:55.690677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb74eb-e8ac-47cf-8229-9c5c24950389 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:55.694313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1299f-5184-4258-bf5b-f0b695b7d998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:55.694920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1299f-5184-4258-bf5b-f0b695b7d998 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:55.695204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1299f-5184-4258-bf5b-f0b695b7d998 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:55.695473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1299f-5184-4258-bf5b-f0b695b7d998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:55.705870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1299f-5184-4258-bf5b-f0b695b7d998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:55.707323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1299f-5184-4258-bf5b-f0b695b7d998 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:55.718800 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1386/2800] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:56.117552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb49e4a-5a5f-4953-abde-c1950ac59cc8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:56.119042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1fb49e4a-5a5f-4953-abde-c1950ac59cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:56.132200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb49e4a-5a5f-4953-abde-c1950ac59cc8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:56.138928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:56.141216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.141649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.142146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] PATH_INFO: `/v3/projects/18370bede7df4cf9b89802805c3885b5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.143744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18370bede7df4cf9b89802805c3885b5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:56.146669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:56.205789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18370bede7df4cf9b89802805c3885b5', 'request_id': 'req-9198f536-b29e-4f03-8ddb-67e3b3fefc39'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:56.215161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18370bede7df4cf9b89802805c3885b5', 'request_id': 'req-9198f536-b29e-4f03-8ddb-67e3b3fefc39'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:56.224306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18370bede7df4cf9b89802805c3885b5', 'request_id': 'req-9198f536-b29e-4f03-8ddb-67e3b3fefc39'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:56.243555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] The token cache is being invalidate because project 18370bede7df4cf9b89802805c3885b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:45:56.244177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:56.245084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9198f536-b29e-4f03-8ddb-67e3b3fefc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:45:56.247106 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1415/2801] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:45:56 2022] DELETE /identity/v3/projects/18370bede7df4cf9b89802805c3885b5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:56.253998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64725321-f26c-4e12-a40d-e83147e6dd10 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.254500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64725321-f26c-4e12-a40d-e83147e6dd10 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.254712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64725321-f26c-4e12-a40d-e83147e6dd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.300552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-64725321-f26c-4e12-a40d-e83147e6dd10 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:56.301940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-64725321-f26c-4e12-a40d-e83147e6dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:56.335572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:56.394246 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1387/2802] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:45:56.399097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e28eb0-9664-4169-a6be-1d7cf18f63da None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.399390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e28eb0-9664-4169-a6be-1d7cf18f63da None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.399666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e28eb0-9664-4169-a6be-1d7cf18f63da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.423147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-20e28eb0-9664-4169-a6be-1d7cf18f63da None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:56.424592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-20e28eb0-9664-4169-a6be-1d7cf18f63da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:56.438096 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1416/2803] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:56.439620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64725321-f26c-4e12-a40d-e83147e6dd10 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:56.450585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64725321-f26c-4e12-a40d-e83147e6dd10 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:56.456109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:56.456717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.457055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.457388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.458072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:45:56.458372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:56.460080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:56.477983 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1388/2804] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:45:56.479199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e28eb0-9664-4169-a6be-1d7cf18f63da None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:56.490621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e28eb0-9664-4169-a6be-1d7cf18f63da None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:56.495687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ab422-1f9f-43d1-839f-04128672a46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:56.496350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ab422-1f9f-43d1-839f-04128672a46d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.496693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ab422-1f9f-43d1-839f-04128672a46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.496945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ab422-1f9f-43d1-839f-04128672a46d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.497601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ab422-1f9f-43d1-839f-04128672a46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:56.499835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ab422-1f9f-43d1-839f-04128672a46d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:56.525067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:56.548059 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1417/2805] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:45:56 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:56.552756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:56.563701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420f549-19ca-49ee-8ca1-87efa6a12787 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:56.639115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457138c3-effb-4e75-a3b4-d555154dbcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:56.640088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457138c3-effb-4e75-a3b4-d555154dbcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.640398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457138c3-effb-4e75-a3b4-d555154dbcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.640675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457138c3-effb-4e75-a3b4-d555154dbcf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.641321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457138c3-effb-4e75-a3b4-d555154dbcf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:56.643431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457138c3-effb-4e75-a3b4-d555154dbcf9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:56.674359 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1389/2806] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:45:56 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:45:56.675612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ab422-1f9f-43d1-839f-04128672a46d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:56.688320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ab422-1f9f-43d1-839f-04128672a46d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:56.694891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89156928-3e09-4e76-98e4-667bdf022506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:56.695679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89156928-3e09-4e76-98e4-667bdf022506 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.695970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89156928-3e09-4e76-98e4-667bdf022506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.696395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89156928-3e09-4e76-98e4-667bdf022506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.697056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89156928-3e09-4e76-98e4-667bdf022506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:56.699218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89156928-3e09-4e76-98e4-667bdf022506 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:56.713251 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1418/2807] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:45:56.718617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457138c3-effb-4e75-a3b4-d555154dbcf9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:56.729578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457138c3-effb-4e75-a3b4-d555154dbcf9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:56.736708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286f8b-ee5b-4dff-86e0-bd233436709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:56.737810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286f8b-ee5b-4dff-86e0-bd233436709a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.738330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286f8b-ee5b-4dff-86e0-bd233436709a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.738833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286f8b-ee5b-4dff-86e0-bd233436709a admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.739642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286f8b-ee5b-4dff-86e0-bd233436709a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:56.741256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286f8b-ee5b-4dff-86e0-bd233436709a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:56.752670 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1390/2808] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:45:56.756749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89156928-3e09-4e76-98e4-667bdf022506 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:56.768433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89156928-3e09-4e76-98e4-667bdf022506 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:56.773399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:56.774130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.774493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.774697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] PATH_INFO: `/v3/projects/9bb2b610f1454b75b098a095eb4f9ec4/users/01b52de389cf46ca9e1af0b5a47da8d3/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.793883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb2b610f1454b75b098a095eb4f9ec4, user_id=01b52de389cf46ca9e1af0b5a47da8d3, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:56.796842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:56.810845 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1419/2809] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:45:56 2022] PUT /identity/v3/projects/9bb2b610f1454b75b098a095eb4f9ec4/users/01b52de389cf46ca9e1af0b5a47da8d3/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:45:56.878047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a3572-2a21-4af9-9187-0eac561d94f9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:56.878391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a3572-2a21-4af9-9187-0eac561d94f9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:56.878802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a3572-2a21-4af9-9187-0eac561d94f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:56.912155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f96a3572-2a21-4af9-9187-0eac561d94f9 None None] MFA Rules not processed for user `01b52de389cf46ca9e1af0b5a47da8d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:45:56.913512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f96a3572-2a21-4af9-9187-0eac561d94f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:56.987220 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1391/2810] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:45:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:45:56.998764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:57.011896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:45:57.105493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599893d5-15f4-40b6-90a8-ee0d189f882c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:57.112501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b6ed-2506-41b6-86ae-0f649cabb30e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:57.113116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b6ed-2506-41b6-86ae-0f649cabb30e service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:57.113392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b6ed-2506-41b6-86ae-0f649cabb30e service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:57.113720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b6ed-2506-41b6-86ae-0f649cabb30e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:57.128101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b6ed-2506-41b6-86ae-0f649cabb30e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:57.129733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b6ed-2506-41b6-86ae-0f649cabb30e service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:45:57.141241 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1420/2811] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:45:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:45:59.888283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a3572-2a21-4af9-9187-0eac561d94f9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:45:59.915785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:45:59.952917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a3572-2a21-4af9-9187-0eac561d94f9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:45:59.957354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:45:59.958019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:45:59.958296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:45:59.958566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] PATH_INFO: `/v3/users/01b52de389cf46ca9e1af0b5a47da8d3` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:45:59.959679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b52de389cf46ca9e1af0b5a47da8d3)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:45:59.961606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:00.007098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b52de389cf46ca9e1af0b5a47da8d3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.013999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b52de389cf46ca9e1af0b5a47da8d3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.015140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b52de389cf46ca9e1af0b5a47da8d3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.015782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b52de389cf46ca9e1af0b5a47da8d3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.022951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b52de389cf46ca9e1af0b5a47da8d3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.040590 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1392/2812] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:45:59 2022] DELETE /identity/v3/users/01b52de389cf46ca9e1af0b5a47da8d3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:46:00.045014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef862c0-fa50-4e6c-8401-10afc094db92 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.045386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef862c0-fa50-4e6c-8401-10afc094db92 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.045589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef862c0-fa50-4e6c-8401-10afc094db92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.067529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3ef862c0-fa50-4e6c-8401-10afc094db92 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:46:00.068743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ef862c0-fa50-4e6c-8401-10afc094db92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:00.129048 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1421/2813] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:46:00.139840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:00.141246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:00.198573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e0b9-4d0f-4224-a3da-fc7f4cb6224f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:00.202266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01592286-4516-4ec4-a75d-506d38a21246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:00.202806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01592286-4516-4ec4-a75d-506d38a21246 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.203144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01592286-4516-4ec4-a75d-506d38a21246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.203360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01592286-4516-4ec4-a75d-506d38a21246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.212187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01592286-4516-4ec4-a75d-506d38a21246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:00.213604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01592286-4516-4ec4-a75d-506d38a21246 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:00.223765 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1393/2814] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:46:00.496880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef862c0-fa50-4e6c-8401-10afc094db92 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:00.498365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ef862c0-fa50-4e6c-8401-10afc094db92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:00.518954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef862c0-fa50-4e6c-8401-10afc094db92 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:00.525063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:00.525827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.526179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.526465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] PATH_INFO: `/v3/projects/9bb2b610f1454b75b098a095eb4f9ec4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.527635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb2b610f1454b75b098a095eb4f9ec4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:00.529567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:00.576474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb2b610f1454b75b098a095eb4f9ec4', 'request_id': 'req-070e86d0-e278-4732-8bb5-641fb888281d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.580637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:00.585367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb2b610f1454b75b098a095eb4f9ec4', 'request_id': 'req-070e86d0-e278-4732-8bb5-641fb888281d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.593488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb2b610f1454b75b098a095eb4f9ec4', 'request_id': 'req-070e86d0-e278-4732-8bb5-641fb888281d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.603731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] The token cache is being invalidate because project 9bb2b610f1454b75b098a095eb4f9ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:46:00.604030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.604637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-070e86d0-e278-4732-8bb5-641fb888281d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:00.606137 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1422/2815] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:46:00 2022] DELETE /identity/v3/projects/9bb2b610f1454b75b098a095eb4f9ec4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:46:00.611094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac1948-be60-4ae3-9bc9-db73c424001c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.611396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac1948-be60-4ae3-9bc9-db73c424001c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.611660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac1948-be60-4ae3-9bc9-db73c424001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.630950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-86ac1948-be60-4ae3-9bc9-db73c424001c None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:46:00.632171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-86ac1948-be60-4ae3-9bc9-db73c424001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:00.674297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:00.693538 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1394/2816] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:46:00.701300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277dd6-9a7d-4586-8029-f47954572edc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.701592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277dd6-9a7d-4586-8029-f47954572edc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.701897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277dd6-9a7d-4586-8029-f47954572edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.722494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0f277dd6-9a7d-4586-8029-f47954572edc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:46:00.723746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0f277dd6-9a7d-4586-8029-f47954572edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:00.731747 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1423/2817] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:46:00.735866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac1948-be60-4ae3-9bc9-db73c424001c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:00.745485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac1948-be60-4ae3-9bc9-db73c424001c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:00.750243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:00.750841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.751123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.751403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.752047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:46:00.752354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:00.754095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:00.766341 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1395/2818] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:46:00.770443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f277dd6-9a7d-4586-8029-f47954572edc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:00.779728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f277dd6-9a7d-4586-8029-f47954572edc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:00.785048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96fb2-104f-4a43-912f-7382dd7757a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:00.785658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96fb2-104f-4a43-912f-7382dd7757a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.785958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96fb2-104f-4a43-912f-7382dd7757a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.786234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96fb2-104f-4a43-912f-7382dd7757a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.786865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e96fb2-104f-4a43-912f-7382dd7757a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:00.788810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e96fb2-104f-4a43-912f-7382dd7757a6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:00.815330 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1424/2819] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:46:00 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:46:00.819519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:00.829919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a4dff-0be3-41ad-b3ec-f5fbfc9d6482 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:00.885306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9ae64-7046-4e53-988e-9f2ddc46c58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:00.885943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9ae64-7046-4e53-988e-9f2ddc46c58b admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.886223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9ae64-7046-4e53-988e-9f2ddc46c58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.886497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9ae64-7046-4e53-988e-9f2ddc46c58b admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.887129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9ae64-7046-4e53-988e-9f2ddc46c58b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:00.889036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9ae64-7046-4e53-988e-9f2ddc46c58b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:00.914404 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1396/2820] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:46:00 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:46:00.915855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96fb2-104f-4a43-912f-7382dd7757a6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:00.926657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96fb2-104f-4a43-912f-7382dd7757a6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:00.931465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfa96c-161c-46be-8330-a1697af0002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:00.932071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfa96c-161c-46be-8330-a1697af0002c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.932355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfa96c-161c-46be-8330-a1697af0002c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.932629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfa96c-161c-46be-8330-a1697af0002c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.933298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfa96c-161c-46be-8330-a1697af0002c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:00.934949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfa96c-161c-46be-8330-a1697af0002c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:00.946136 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1425/2821] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:46:00.950160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9ae64-7046-4e53-988e-9f2ddc46c58b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:00.961441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9ae64-7046-4e53-988e-9f2ddc46c58b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:00.967122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1dac9-0bf9-4627-8e05-63b9b3dafd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:00.967779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1dac9-0bf9-4627-8e05-63b9b3dafd3e admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:00.968709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1dac9-0bf9-4627-8e05-63b9b3dafd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:00.968996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1dac9-0bf9-4627-8e05-63b9b3dafd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:00.969681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1dac9-0bf9-4627-8e05-63b9b3dafd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:00.971391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1dac9-0bf9-4627-8e05-63b9b3dafd3e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:00.983202 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1397/2822] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:46:00.987951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfa96c-161c-46be-8330-a1697af0002c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:01.002535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfa96c-161c-46be-8330-a1697af0002c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:01.007312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:01.008056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:01.008405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:01.008684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] PATH_INFO: `/v3/projects/e9899a5ea07b408e8614ec4e33260a87/users/b68705385eff4796874799b53044ba5e/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:01.026083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9899a5ea07b408e8614ec4e33260a87, user_id=b68705385eff4796874799b53044ba5e, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:01.028771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:01.043206 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1426/2823] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:46:00 2022] PUT /identity/v3/projects/e9899a5ea07b408e8614ec4e33260a87/users/b68705385eff4796874799b53044ba5e/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:46:01.054979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfa48-deae-4dd9-a5bc-22659369d91e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:01.055272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfa48-deae-4dd9-a5bc-22659369d91e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:01.055551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfa48-deae-4dd9-a5bc-22659369d91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:01.084854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5cabfa48-deae-4dd9-a5bc-22659369d91e None None] MFA Rules not processed for user `b68705385eff4796874799b53044ba5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:46:01.086104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5cabfa48-deae-4dd9-a5bc-22659369d91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:01.149668 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1398/2824] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:46:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:46:01.160374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:01.162180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:01.218453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902b3d0-4d72-4a6b-adab-7a2de27c539e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:01.222074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbfb995-9167-4e46-8a26-7fbdb3bd8bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:01.222670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfb995-9167-4e46-8a26-7fbdb3bd8bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:01.222953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfb995-9167-4e46-8a26-7fbdb3bd8bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:01.223226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbfb995-9167-4e46-8a26-7fbdb3bd8bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:01.231951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfb995-9167-4e46-8a26-7fbdb3bd8bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:01.234052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbfb995-9167-4e46-8a26-7fbdb3bd8bb1 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:01.242735 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1427/2825] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:46:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:46:11.146766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:11.238787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=59,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:11.245130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabfa48-deae-4dd9-a5bc-22659369d91e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:11.307463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabfa48-deae-4dd9-a5bc-22659369d91e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:11.312014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:11.312631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:11.312906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:11.313174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] PATH_INFO: `/v3/users/b68705385eff4796874799b53044ba5e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:11.314461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68705385eff4796874799b53044ba5e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:11.316362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:11.360394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68705385eff4796874799b53044ba5e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.366986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68705385eff4796874799b53044ba5e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.367636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68705385eff4796874799b53044ba5e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.368245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68705385eff4796874799b53044ba5e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.375511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68705385eff4796874799b53044ba5e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.393915 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1399/2826] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:46:11 2022] DELETE /identity/v3/users/b68705385eff4796874799b53044ba5e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:46:11.398472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b71ce-e32f-4f91-9627-c58d192ca493 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:11.398756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b71ce-e32f-4f91-9627-c58d192ca493 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:11.399056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b71ce-e32f-4f91-9627-c58d192ca493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:11.419057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c09b71ce-e32f-4f91-9627-c58d192ca493 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:46:11.420261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c09b71ce-e32f-4f91-9627-c58d192ca493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:11.482690 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1428/2827] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:46:11.493258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:11.494688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:11.551217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1c2d-6fed-4f48-8a05-0077fbc44729 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:11.554749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:11.555349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:11.555631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:11.555905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:11.565680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:11.567113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:11.576368 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1400/2828] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:46:11.841781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b71ce-e32f-4f91-9627-c58d192ca493 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:46:11.843549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c09b71ce-e32f-4f91-9627-c58d192ca493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:46:11.861279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b71ce-e32f-4f91-9627-c58d192ca493 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:46:11.866146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:46:11.866858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:46:11.867052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:46:11.867439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] PATH_INFO: `/v3/projects/e9899a5ea07b408e8614ec4e33260a87` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:46:11.868449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9899a5ea07b408e8614ec4e33260a87)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:46:11.870427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:46:11.917848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9899a5ea07b408e8614ec4e33260a87', 'request_id': 'req-5bf28664-b47c-48e1-a824-114337f00478'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.925460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9899a5ea07b408e8614ec4e33260a87', 'request_id': 'req-5bf28664-b47c-48e1-a824-114337f00478'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.931703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9899a5ea07b408e8614ec4e33260a87', 'request_id': 'req-5bf28664-b47c-48e1-a824-114337f00478'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.942801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] The token cache is being invalidate because project e9899a5ea07b408e8614ec4e33260a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:46:11.943086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.943737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:46:11.945218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1429/2829] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:46:11 2022] DELETE /identity/v3/projects/e9899a5ea07b408e8614ec4e33260a87 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:46:21.572305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:46:21.939806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:09.272313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:47:09.273850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:47:09.284923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b520ea-96cf-4a2e-8b80-96c72a956b9c service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:47:09.333424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:47:09.333973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:47:09.334251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:47:09.334522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:47:09.336210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:47:09.345538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:47:09.346989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:47:09.354877 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1401/2830] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:47:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:47:19.351318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=26 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:25.396816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:47:25.398244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:47:25.454033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf28664-b47c-48e1-a824-114337f00478 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:47:25.457998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:47:25.458601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:47:25.458881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:47:25.459148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:47:25.460874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:47:25.529114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:47:25.530580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:47:25.542057 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1430/2831] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:47:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:47:29.369934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:47:29.371327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:47:29.382184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7b967-f88a-4e25-9238-2623cfcfab96 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:47:29.385782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:47:29.386371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:47:29.386647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:47:29.386916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:47:29.388575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:47:29.402821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:47:29.404246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:47:29.416477 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1402/2832] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:47:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:47:35.537229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=54 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:47:39.411449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:06.415897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:50:06.417503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:06.439527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58353ac-6ccf-4425-9de8-0e44f64cc926 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:50:06.446183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8afd24-a760-4adc-921c-204354f52efb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:50:06.446793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8afd24-a760-4adc-921c-204354f52efb service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:50:06.447070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8afd24-a760-4adc-921c-204354f52efb service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:50:06.447349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8afd24-a760-4adc-921c-204354f52efb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:50:06.449234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bf8afd24-a760-4adc-921c-204354f52efb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:06.464277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8afd24-a760-4adc-921c-204354f52efb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:50:06.465926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8afd24-a760-4adc-921c-204354f52efb service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:50:06.478223 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1431/2833] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:50:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:50:06.593751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:50:06.595690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:06.683780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19b296-6d65-4f39-83d3-d93fdd32d569 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:50:06.688751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:50:06.689371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:50:06.689828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:50:06.690108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:50:06.691860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:06.704105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:50:06.705791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:50:06.717048 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1403/2834] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:50:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:50:09.558590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b706-3969-480c-8620-b2faae2547c1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:50:09.559116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b706-3969-480c-8620-b2faae2547c1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:50:09.559116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b706-3969-480c-8620-b2faae2547c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:50:09.578597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6fd5b706-3969-480c-8620-b2faae2547c1 None None] MFA Rules not processed for user `aec22e5f030d43fe8de90d17fe8225dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:50:09.579823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6fd5b706-3969-480c-8620-b2faae2547c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:09.634347 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1432/2835] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:50:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:50:09.937112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:50:09.946648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebe13c-9d46-4f95-9432-d6f05b7056d6 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:50:09.950218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:50:09.950802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:50:09.951079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:50:09.951348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:50:09.953026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:09.962524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:50:09.963926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:50:09.971750 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1404/2836] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:50:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:50:10.306028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5b706-3969-480c-8620-b2faae2547c1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:50:10.307427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6fd5b706-3969-480c-8620-b2faae2547c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:10.322422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5b706-3969-480c-8620-b2faae2547c1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:50:10.327393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:50:10.328008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:50:10.328290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:50:10.328562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:50:10.330351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:10.343016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:50:10.344619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:50:10.355423 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1433/2837] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:50:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:50:19.465613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:50:19.467394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:19.486238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b1ed-15cc-4110-a50a-a2e234b25676 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:50:19.491747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:50:19.492346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:50:19.492620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:50:19.492887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:50:19.494718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:19.506831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:50:19.508417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:50:19.519809 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1405/2838] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:50:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:50:20.350358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=37 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:23.040102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:50:23.052937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1cfec-c52a-44f3-a73c-561752d395a9 service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:50:23.059024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:50:23.059699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:50:23.060057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:50:23.060414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:50:23.062295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:50:23.075375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:50:23.076976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:50:23.087866 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1434/2839] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:50:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:50:29.515411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=44 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:50:33.083092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:38.011786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:51:38.023344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085dbfa-b0e4-444b-847e-bda030be6139 service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:51:38.028772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:51:38.029504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:51:38.029828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:51:38.030109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:51:38.031839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:51:38.041951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:51:38.043519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:51:38.053076 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1406/2840] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:51:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:51:48.048984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:51:58.289866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:51:58.300428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68318d-64fe-4f92-8c59-983b6434ddda service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:51:58.303935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:51:58.304544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:51:58.304821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:51:58.305087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:51:58.306819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:51:58.374640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:51:58.376696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:51:58.386680 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1435/2841] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:01.198195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:01.211384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cbc3d-d026-4098-a92a-4c00b459a69d service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:01.216408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:01.217190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:01.217579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:01.217979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:01.220379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:01.232876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:01.234830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:01.245876 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1407/2842] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:01.276373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:01.280151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:01.298029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb146819-c0c8-4a8c-88e3-8e90508f671e service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:01.302848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:01.303524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:01.304060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:01.304400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] PATH_INFO: `/v3/users/aec22e5f030d43fe8de90d17fe8225dc` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:01.305967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec22e5f030d43fe8de90d17fe8225dc)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:01.307961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:01.367320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec22e5f030d43fe8de90d17fe8225dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.374369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec22e5f030d43fe8de90d17fe8225dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.375753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec22e5f030d43fe8de90d17fe8225dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.376562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec22e5f030d43fe8de90d17fe8225dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.384943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-254772d2-06d7-4628-841c-cc82a3daa928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec22e5f030d43fe8de90d17fe8225dc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.405934 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1436/2843] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:52:01 2022] DELETE /identity/v3/users/aec22e5f030d43fe8de90d17fe8225dc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:01.579288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:01.591816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49bad8-76d8-4a19-8d44-8590a89d58d8 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:01.649176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:01.649834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:01.650111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:01.650377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] PATH_INFO: `/v3/projects/319bd365127d481aac9f127196321ecf` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:01.651478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=319bd365127d481aac9f127196321ecf)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:01.653360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:01.698477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '319bd365127d481aac9f127196321ecf', 'request_id': 'req-bcc849d0-ed23-4046-8b20-02339597d9bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.706007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '319bd365127d481aac9f127196321ecf', 'request_id': 'req-bcc849d0-ed23-4046-8b20-02339597d9bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.712345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '319bd365127d481aac9f127196321ecf', 'request_id': 'req-bcc849d0-ed23-4046-8b20-02339597d9bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.723598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] The token cache is being invalidate because project 319bd365127d481aac9f127196321ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:52:01.723880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.724508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bcc849d0-ed23-4046-8b20-02339597d9bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:01.726015 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1408/2844] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:52:01 2022] DELETE /identity/v3/projects/319bd365127d481aac9f127196321ecf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:01.730940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcd7c6-1957-457b-9727-3a1406a51f1c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:01.731232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcd7c6-1957-457b-9727-3a1406a51f1c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:01.731511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcd7c6-1957-457b-9727-3a1406a51f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:01.752148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-36bcd7c6-1957-457b-9727-3a1406a51f1c None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:01.753350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-36bcd7c6-1957-457b-9727-3a1406a51f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:01.812929 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1437/2845] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:01.820486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655d51-cb89-4e9b-b1fa-8be3ac927340 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:01.820775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655d51-cb89-4e9b-b1fa-8be3ac927340 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:01.821061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce655d51-cb89-4e9b-b1fa-8be3ac927340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:01.841081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ce655d51-cb89-4e9b-b1fa-8be3ac927340 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:01.842320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ce655d51-cb89-4e9b-b1fa-8be3ac927340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:01.850874 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1409/2846] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:01.855118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcd7c6-1957-457b-9727-3a1406a51f1c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:01.864749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcd7c6-1957-457b-9727-3a1406a51f1c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:01.869373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:01.870010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:01.870295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:01.870571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:01.871221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:52:01.871528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:01.873138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:01.874612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:01.886169 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1438/2847] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:52:01.890333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce655d51-cb89-4e9b-b1fa-8be3ac927340 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:01.900071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce655d51-cb89-4e9b-b1fa-8be3ac927340 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:01.904826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb097727-a50a-4408-ba22-1e7292488dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:01.905420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097727-a50a-4408-ba22-1e7292488dfd admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:01.905732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097727-a50a-4408-ba22-1e7292488dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:01.906005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097727-a50a-4408-ba22-1e7292488dfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:01.907222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb097727-a50a-4408-ba22-1e7292488dfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:01.909137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb097727-a50a-4408-ba22-1e7292488dfd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:01.935460 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1410/2848] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:52:01 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 30 09:52:01.939532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:01.949301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255244dd-2456-4c45-b94d-74a3b692676a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:02.006289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c0736-010f-4271-ada3-7750c558e5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:02.006891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c0736-010f-4271-ada3-7750c558e5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.007173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c0736-010f-4271-ada3-7750c558e5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.007445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c0736-010f-4271-ada3-7750c558e5b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.008078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c0736-010f-4271-ada3-7750c558e5b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:02.010043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c0736-010f-4271-ada3-7750c558e5b5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:02.030489 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1439/2849] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:52:01 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:52:02.034796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb097727-a50a-4408-ba22-1e7292488dfd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:02.045367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb097727-a50a-4408-ba22-1e7292488dfd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:02.050566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5c666-a853-45f9-8f6e-b26108d5a4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:02.051170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5c666-a853-45f9-8f6e-b26108d5a4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.051451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5c666-a853-45f9-8f6e-b26108d5a4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.051729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5c666-a853-45f9-8f6e-b26108d5a4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.052379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5c666-a853-45f9-8f6e-b26108d5a4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:02.054280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5c666-a853-45f9-8f6e-b26108d5a4fd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:02.065324 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1411/2850] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:52:02.069141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c0736-010f-4271-ada3-7750c558e5b5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:02.079154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c0736-010f-4271-ada3-7750c558e5b5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:02.083970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367cf082-7b3a-41d2-b784-9d42176fae53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:02.084648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cf082-7b3a-41d2-b784-9d42176fae53 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.084964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cf082-7b3a-41d2-b784-9d42176fae53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.085430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cf082-7b3a-41d2-b784-9d42176fae53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.086194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367cf082-7b3a-41d2-b784-9d42176fae53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:02.088195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367cf082-7b3a-41d2-b784-9d42176fae53 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:02.098842 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1440/2851] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 30 09:52:02.102817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5c666-a853-45f9-8f6e-b26108d5a4fd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:02.113357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5c666-a853-45f9-8f6e-b26108d5a4fd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:02.118399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:02.119048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.119353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.119625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] PATH_INFO: `/v3/projects/024574053dab49e29b04727d54b269f2/users/ac9480c04022442fb576b886da928c07/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.134346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=024574053dab49e29b04727d54b269f2, user_id=ac9480c04022442fb576b886da928c07, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:02.136969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:02.149685 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1412/2852] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:52:02 2022] PUT /identity/v3/projects/024574053dab49e29b04727d54b269f2/users/ac9480c04022442fb576b886da928c07/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:02.161587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20836d-b6af-491d-a9a4-02239d5bfd2d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.161900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20836d-b6af-491d-a9a4-02239d5bfd2d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.162174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20836d-b6af-491d-a9a4-02239d5bfd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.190589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-eb20836d-b6af-491d-a9a4-02239d5bfd2d None None] MFA Rules not processed for user `ac9480c04022442fb576b886da928c07`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:02.191784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eb20836d-b6af-491d-a9a4-02239d5bfd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:02.257492 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1441/2853] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:02.269428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:02.271147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:02.286832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:02.348080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e7804-1333-4dbd-9ad1-fd8184831822 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:02.353939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197f816-c30a-4793-9fa7-6a767204122a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:02.354551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f816-c30a-4793-9fa7-6a767204122a service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.354835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f816-c30a-4793-9fa7-6a767204122a service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.355113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197f816-c30a-4793-9fa7-6a767204122a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.363662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197f816-c30a-4793-9fa7-6a767204122a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:02.365265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197f816-c30a-4793-9fa7-6a767204122a service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:02.374081 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1413/2854] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:02.579224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20836d-b6af-491d-a9a4-02239d5bfd2d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:02.641760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20836d-b6af-491d-a9a4-02239d5bfd2d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:02.646211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:02.646827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.647102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.647367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] PATH_INFO: `/v3/users/ac9480c04022442fb576b886da928c07` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.648468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9480c04022442fb576b886da928c07)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:02.650439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:02.652634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:02.699233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9480c04022442fb576b886da928c07'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:02.706578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9480c04022442fb576b886da928c07'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:02.707239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9480c04022442fb576b886da928c07'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:02.707861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9480c04022442fb576b886da928c07'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:02.715137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9480c04022442fb576b886da928c07'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:02.732282 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1442/2855] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:52:02 2022] DELETE /identity/v3/users/ac9480c04022442fb576b886da928c07 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:02.737488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62f6ba-9507-49db-97c0-3b9c9e58bb2e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.737829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62f6ba-9507-49db-97c0-3b9c9e58bb2e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.738131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62f6ba-9507-49db-97c0-3b9c9e58bb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.757550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5e62f6ba-9507-49db-97c0-3b9c9e58bb2e None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:02.758780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5e62f6ba-9507-49db-97c0-3b9c9e58bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:02.816536 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1414/2856] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:02.826327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:02.827733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:02.885736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf065b9-da8e-475b-ad1a-11505fb96684 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:02.889350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282ac0b-3d61-4071-9dc7-2e769cddd40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:02.890235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282ac0b-3d61-4071-9dc7-2e769cddd40f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:02.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282ac0b-3d61-4071-9dc7-2e769cddd40f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:02.891975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282ac0b-3d61-4071-9dc7-2e769cddd40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:02.900839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282ac0b-3d61-4071-9dc7-2e769cddd40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:02.902310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282ac0b-3d61-4071-9dc7-2e769cddd40f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:02.912168 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1443/2857] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:03.179925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62f6ba-9507-49db-97c0-3b9c9e58bb2e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.181469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5e62f6ba-9507-49db-97c0-3b9c9e58bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:03.193598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62f6ba-9507-49db-97c0-3b9c9e58bb2e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.198445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.199086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.199362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.199638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] PATH_INFO: `/v3/projects/024574053dab49e29b04727d54b269f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.200772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=024574053dab49e29b04727d54b269f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.202722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:03.250435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024574053dab49e29b04727d54b269f2', 'request_id': 'req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:03.257658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024574053dab49e29b04727d54b269f2', 'request_id': 'req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:03.263833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024574053dab49e29b04727d54b269f2', 'request_id': 'req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:03.275019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] The token cache is being invalidate because project 024574053dab49e29b04727d54b269f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:52:03.275617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:03.275925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3687f94a-e9c4-42e9-a098-9abea5b9e7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:03.277428 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1415/2858] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:52:03 2022] DELETE /identity/v3/projects/024574053dab49e29b04727d54b269f2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:03.282836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be996ad-34be-4b38-b83f-435dbac8f39a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.283122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be996ad-34be-4b38-b83f-435dbac8f39a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.283395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be996ad-34be-4b38-b83f-435dbac8f39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.302453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6be996ad-34be-4b38-b83f-435dbac8f39a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:03.303649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6be996ad-34be-4b38-b83f-435dbac8f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:03.362193 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1444/2859] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:03.370125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529f696-9beb-4d2f-88ac-31841e1bae69 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.370397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529f696-9beb-4d2f-88ac-31841e1bae69 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.370667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529f696-9beb-4d2f-88ac-31841e1bae69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.391041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0529f696-9beb-4d2f-88ac-31841e1bae69 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:03.392391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0529f696-9beb-4d2f-88ac-31841e1bae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:03.400249 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1416/2860] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:03.404485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be996ad-34be-4b38-b83f-435dbac8f39a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.415070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be996ad-34be-4b38-b83f-435dbac8f39a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.419572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.420169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.420449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.420725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.421373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:52:03.421767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.423471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:03.434686 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1445/2861] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:52:03.439484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529f696-9beb-4d2f-88ac-31841e1bae69 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.448639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529f696-9beb-4d2f-88ac-31841e1bae69 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.453847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbb5b2-f0fa-41c9-b1e5-48ac6be8762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.454441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb5b2-f0fa-41c9-b1e5-48ac6be8762d admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.454735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb5b2-f0fa-41c9-b1e5-48ac6be8762d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.454984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbb5b2-f0fa-41c9-b1e5-48ac6be8762d admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.455612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcbb5b2-f0fa-41c9-b1e5-48ac6be8762d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.457821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcbb5b2-f0fa-41c9-b1e5-48ac6be8762d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:03.475047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:03.489006 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1417/2862] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:52:03 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:52:03.493079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.503178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0f0be-7bff-46f4-ac16-d090cdc36bf2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.511207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:03.561784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eda764-2a0e-4297-af3b-b9cb865a884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.562474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eda764-2a0e-4297-af3b-b9cb865a884c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.562967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eda764-2a0e-4297-af3b-b9cb865a884c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.563123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eda764-2a0e-4297-af3b-b9cb865a884c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.563759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eda764-2a0e-4297-af3b-b9cb865a884c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.565734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eda764-2a0e-4297-af3b-b9cb865a884c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:03.586956 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1446/2863] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:52:03 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:52:03.591030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbb5b2-f0fa-41c9-b1e5-48ac6be8762d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.602032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbb5b2-f0fa-41c9-b1e5-48ac6be8762d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.607020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24da646-7e3a-460e-bcc2-6bfe9c27d3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.607609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24da646-7e3a-460e-bcc2-6bfe9c27d3ca admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.607885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24da646-7e3a-460e-bcc2-6bfe9c27d3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.608159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24da646-7e3a-460e-bcc2-6bfe9c27d3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.608802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24da646-7e3a-460e-bcc2-6bfe9c27d3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.610486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24da646-7e3a-460e-bcc2-6bfe9c27d3ca admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:03.620027 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1418/2864] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:52:03.623920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eda764-2a0e-4297-af3b-b9cb865a884c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.634503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eda764-2a0e-4297-af3b-b9cb865a884c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.639066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b58213-8a49-4426-a546-2350a1db031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.639649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58213-8a49-4426-a546-2350a1db031e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.640185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58213-8a49-4426-a546-2350a1db031e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.640584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b58213-8a49-4426-a546-2350a1db031e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.642098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b58213-8a49-4426-a546-2350a1db031e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.643700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b58213-8a49-4426-a546-2350a1db031e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:03.654013 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1447/2865] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:52:03.658637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24da646-7e3a-460e-bcc2-6bfe9c27d3ca admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.668460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24da646-7e3a-460e-bcc2-6bfe9c27d3ca admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.674045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.674629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.674910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.675203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] PATH_INFO: `/v3/projects/f7a0f967f46c4db08aa76d924045ee30/users/20b35763ec974f728cb983e9cfe5ab64/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.689952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a0f967f46c4db08aa76d924045ee30, user_id=20b35763ec974f728cb983e9cfe5ab64, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.693026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:03.707372 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1419/2866] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:52:03 2022] PUT /identity/v3/projects/f7a0f967f46c4db08aa76d924045ee30/users/20b35763ec974f728cb983e9cfe5ab64/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:03.717751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a01ef-d31b-451d-b20d-2681f19173e2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.718039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a01ef-d31b-451d-b20d-2681f19173e2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.718310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a01ef-d31b-451d-b20d-2681f19173e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.746344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-993a01ef-d31b-451d-b20d-2681f19173e2 None None] MFA Rules not processed for user `20b35763ec974f728cb983e9cfe5ab64`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:03.747536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-993a01ef-d31b-451d-b20d-2681f19173e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:03.809695 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1448/2867] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:03.820719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.822549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:03.879105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bcb1a-54e2-41d4-a6d5-c55e7611b549 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.882777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf337ac-c073-4ad5-b409-72630b224a59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.883377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf337ac-c073-4ad5-b409-72630b224a59 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.883660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf337ac-c073-4ad5-b409-72630b224a59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.883935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf337ac-c073-4ad5-b409-72630b224a59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.892359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf337ac-c073-4ad5-b409-72630b224a59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.893815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf337ac-c073-4ad5-b409-72630b224a59 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:03.902757 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1420/2868] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:03.925750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a01ef-d31b-451d-b20d-2681f19173e2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:03.987205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a01ef-d31b-451d-b20d-2681f19173e2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:03.991882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:03.992504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:03.992787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:03.993063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] PATH_INFO: `/v3/users/20b35763ec974f728cb983e9cfe5ab64` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:03.994220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b35763ec974f728cb983e9cfe5ab64)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:03.996150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:04.019272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:04.045367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b35763ec974f728cb983e9cfe5ab64'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.052516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b35763ec974f728cb983e9cfe5ab64'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.053209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b35763ec974f728cb983e9cfe5ab64'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.053868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b35763ec974f728cb983e9cfe5ab64'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.061362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b35763ec974f728cb983e9cfe5ab64'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.079486 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1449/2869] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:52:03 2022] DELETE /identity/v3/users/20b35763ec974f728cb983e9cfe5ab64 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:04.083907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b246c-fb08-4a0e-903f-df6019f1b76b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.084195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b246c-fb08-4a0e-903f-df6019f1b76b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.084474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b246c-fb08-4a0e-903f-df6019f1b76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.103776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-dc5b246c-fb08-4a0e-903f-df6019f1b76b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:04.105014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dc5b246c-fb08-4a0e-903f-df6019f1b76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:04.163182 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1421/2870] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:04.173289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:04.174958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:04.232101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a2708-d164-4ef5-9fd1-f4c35a6ca1b6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:04.236480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdb2d1-e5bb-407b-a55b-c1230a03aa50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:04.237085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb2d1-e5bb-407b-a55b-c1230a03aa50 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.237368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb2d1-e5bb-407b-a55b-c1230a03aa50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.237667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb2d1-e5bb-407b-a55b-c1230a03aa50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.246623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdb2d1-e5bb-407b-a55b-c1230a03aa50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:04.248065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdb2d1-e5bb-407b-a55b-c1230a03aa50 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:04.258075 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1450/2871] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:04.528799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b246c-fb08-4a0e-903f-df6019f1b76b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:04.530524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dc5b246c-fb08-4a0e-903f-df6019f1b76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:04.547530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b246c-fb08-4a0e-903f-df6019f1b76b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:04.553294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:04.553997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.554296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.554485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] PATH_INFO: `/v3/projects/f7a0f967f46c4db08aa76d924045ee30` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.555627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a0f967f46c4db08aa76d924045ee30)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:04.557541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:04.609493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a0f967f46c4db08aa76d924045ee30', 'request_id': 'req-b678c651-4c4e-488c-9c40-779b26a81366'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.617239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a0f967f46c4db08aa76d924045ee30', 'request_id': 'req-b678c651-4c4e-488c-9c40-779b26a81366'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.618414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:04.626240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a0f967f46c4db08aa76d924045ee30', 'request_id': 'req-b678c651-4c4e-488c-9c40-779b26a81366'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.637508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] The token cache is being invalidate because project f7a0f967f46c4db08aa76d924045ee30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:52:04.637839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.638480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b678c651-4c4e-488c-9c40-779b26a81366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:52:04.640210 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1422/2872] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:52:04 2022] DELETE /identity/v3/projects/f7a0f967f46c4db08aa76d924045ee30 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:04.645941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce93f-dc1c-4f21-96e5-1f7a3a00a26c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.646400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce93f-dc1c-4f21-96e5-1f7a3a00a26c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.646500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ce93f-dc1c-4f21-96e5-1f7a3a00a26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.668619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f74ce93f-dc1c-4f21-96e5-1f7a3a00a26c None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:04.669866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f74ce93f-dc1c-4f21-96e5-1f7a3a00a26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:04.730234 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1451/2873] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:04.740264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03397d34-eb2b-4d01-b216-4be5d16d4ced None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.740568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03397d34-eb2b-4d01-b216-4be5d16d4ced None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.740832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03397d34-eb2b-4d01-b216-4be5d16d4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.760658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-03397d34-eb2b-4d01-b216-4be5d16d4ced None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:04.761906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-03397d34-eb2b-4d01-b216-4be5d16d4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:04.770604 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1423/2874] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:04.774697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce93f-dc1c-4f21-96e5-1f7a3a00a26c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:04.784985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ce93f-dc1c-4f21-96e5-1f7a3a00a26c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:04.789691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:04.790293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.790575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.790846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.791487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:52:04.791789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:04.793392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:04.804742 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1452/2875] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:52:04.808829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03397d34-eb2b-4d01-b216-4be5d16d4ced None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:04.818724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03397d34-eb2b-4d01-b216-4be5d16d4ced None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:04.823410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b79e5f-1517-424f-8033-d12917fcf05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:04.824007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b79e5f-1517-424f-8033-d12917fcf05b admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.824287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b79e5f-1517-424f-8033-d12917fcf05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.824560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b79e5f-1517-424f-8033-d12917fcf05b admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.825200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b79e5f-1517-424f-8033-d12917fcf05b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:04.827140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b79e5f-1517-424f-8033-d12917fcf05b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:04.854985 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1424/2876] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:52:04 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:52:04.859061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:04.871618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1b861-ad7d-4ebe-84a7-0a4c0abb97b0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:04.898800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:04.931155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c06ee-e1fb-4944-a42e-71b9b31b9382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:04.931757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c06ee-e1fb-4944-a42e-71b9b31b9382 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.932147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c06ee-e1fb-4944-a42e-71b9b31b9382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.932426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c06ee-e1fb-4944-a42e-71b9b31b9382 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.933056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c06ee-e1fb-4944-a42e-71b9b31b9382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:04.935076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c06ee-e1fb-4944-a42e-71b9b31b9382 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:04.956137 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1453/2877] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:52:04 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:52:04.960231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b79e5f-1517-424f-8033-d12917fcf05b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:04.971033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b79e5f-1517-424f-8033-d12917fcf05b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:04.975565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b159ac-d0d9-4e58-8b87-c455900a0dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:04.976154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b159ac-d0d9-4e58-8b87-c455900a0dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:04.976428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b159ac-d0d9-4e58-8b87-c455900a0dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:04.976698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b159ac-d0d9-4e58-8b87-c455900a0dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:04.977355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b159ac-d0d9-4e58-8b87-c455900a0dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:04.979044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b159ac-d0d9-4e58-8b87-c455900a0dd6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:04.988914 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1425/2878] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:52:04.992645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c06ee-e1fb-4944-a42e-71b9b31b9382 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:05.002616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c06ee-e1fb-4944-a42e-71b9b31b9382 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:05.007522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ffc2a-ff57-412d-b633-07c446054484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:05.008117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ffc2a-ff57-412d-b633-07c446054484 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:05.008399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ffc2a-ff57-412d-b633-07c446054484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:05.008672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ffc2a-ff57-412d-b633-07c446054484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:05.009307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ffc2a-ff57-412d-b633-07c446054484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:05.010959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ffc2a-ff57-412d-b633-07c446054484 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:05.021712 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1454/2879] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:52:05.025723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b159ac-d0d9-4e58-8b87-c455900a0dd6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:05.036417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b159ac-d0d9-4e58-8b87-c455900a0dd6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:05.040921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:05.041554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:05.041866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:05.042139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] PATH_INFO: `/v3/projects/95afd46a9b5947f9918007a23dffdbce/users/7bd88623aa1e43e19db56b4478e1e0ad/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:05.056218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95afd46a9b5947f9918007a23dffdbce, user_id=7bd88623aa1e43e19db56b4478e1e0ad, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:05.058877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:05.072143 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1426/2880] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:52:05 2022] PUT /identity/v3/projects/95afd46a9b5947f9918007a23dffdbce/users/7bd88623aa1e43e19db56b4478e1e0ad/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:52:05.077127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064c71d-9c68-45b2-bc35-15d5886addcd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:05.077471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064c71d-9c68-45b2-bc35-15d5886addcd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:05.077785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064c71d-9c68-45b2-bc35-15d5886addcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:05.097954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1064c71d-9c68-45b2-bc35-15d5886addcd None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:05.099165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1064c71d-9c68-45b2-bc35-15d5886addcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:05.158810 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1455/2881] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:05.170261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:05.172396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:05.233538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056e827-4bf5-4b83-91a9-0ea16a39bf47 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:05.238156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:05.238766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:05.239102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:05.239348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:05.248372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:05.249914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:05.259279 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1427/2882] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:05.908847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8de1a-6149-48ed-97cc-47b9cc4104b7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:05.909423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8de1a-6149-48ed-97cc-47b9cc4104b7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:05.909423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8de1a-6149-48ed-97cc-47b9cc4104b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:05.938667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e6b8de1a-6149-48ed-97cc-47b9cc4104b7 None None] MFA Rules not processed for user `7bd88623aa1e43e19db56b4478e1e0ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:05.939887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e6b8de1a-6149-48ed-97cc-47b9cc4104b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:06.004866 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1456/2883] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:06.015979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:06.017543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:06.076131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70ec5b-9f9f-4025-963f-36d61e2134a9 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:06.080684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:06.081500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:06.082552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:06.082830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:06.091668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:06.093088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:06.103045 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1428/2884] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:07.463125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c91709-7600-43b5-9742-c5899994f743 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:07.463634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c91709-7600-43b5-9742-c5899994f743 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:07.463752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c91709-7600-43b5-9742-c5899994f743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:07.483238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-94c91709-7600-43b5-9742-c5899994f743 None None] MFA Rules not processed for user `7bd88623aa1e43e19db56b4478e1e0ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:07.484501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-94c91709-7600-43b5-9742-c5899994f743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:07.491302 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1457/2885] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:52:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:09.771589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:09.773010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:09.785350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a125b-b4b6-420d-9b7b-0b1a4582b77f service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:09.789388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e12fb0-d032-4d1d-813b-f628191d1a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:09.790033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e12fb0-d032-4d1d-813b-f628191d1a66 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:09.790312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e12fb0-d032-4d1d-813b-f628191d1a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:09.790581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e12fb0-d032-4d1d-813b-f628191d1a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:09.792268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-29e12fb0-d032-4d1d-813b-f628191d1a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:09.801682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e12fb0-d032-4d1d-813b-f628191d1a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:09.803088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e12fb0-d032-4d1d-813b-f628191d1a66 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:09.810894 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1429/2886] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:17.478418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:19.807288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:52:38.565000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c91709-7600-43b5-9742-c5899994f743 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:38.567712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-94c91709-7600-43b5-9742-c5899994f743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:38.580865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c91709-7600-43b5-9742-c5899994f743 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:38.585002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:38.585593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:38.585904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:38.586177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:38.588036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:38.660124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:38.661553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:38.672427 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1458/2887] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:41.640980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a139dc-bf1d-451b-8fff-aa80c2e1aa97 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:41.641484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a139dc-bf1d-451b-8fff-aa80c2e1aa97 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:41.641595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a139dc-bf1d-451b-8fff-aa80c2e1aa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:41.661101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f0a139dc-bf1d-451b-8fff-aa80c2e1aa97 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:41.663065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f0a139dc-bf1d-451b-8fff-aa80c2e1aa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:41.670747 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1430/2888] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:41.691135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:41.692297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:41.707144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c4d11-8681-4977-9d19-580c86627fb0 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:41.713280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:41.713922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:41.714204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:41.714475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:41.722680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:41.724280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:41.733681 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1459/2889] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:44.712342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8290-029f-4b29-b04e-61c07ca3ace8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:44.712886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8290-029f-4b29-b04e-61c07ca3ace8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:44.712886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8290-029f-4b29-b04e-61c07ca3ace8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:44.735417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-03da8290-029f-4b29-b04e-61c07ca3ace8 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:44.736616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-03da8290-029f-4b29-b04e-61c07ca3ace8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:44.746576 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1431/2890] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:44.767890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:44.769321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:44.784423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc74f9-0483-4672-8b52-2d60c7ca6d9e service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:44.790058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860cee-3cec-49f2-9808-72d241dd09a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:44.790649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860cee-3cec-49f2-9808-72d241dd09a6 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:44.790928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860cee-3cec-49f2-9808-72d241dd09a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:44.791200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97860cee-3cec-49f2-9808-72d241dd09a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:44.799362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97860cee-3cec-49f2-9808-72d241dd09a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:44.801014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97860cee-3cec-49f2-9808-72d241dd09a6 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:44.810410 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1460/2891] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:47.945117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f93b0-dd4f-46f1-aaab-61f3a1d554f4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:47.945714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f93b0-dd4f-46f1-aaab-61f3a1d554f4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:47.945714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f93b0-dd4f-46f1-aaab-61f3a1d554f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:47.965571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-163f93b0-dd4f-46f1-aaab-61f3a1d554f4 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:47.966809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-163f93b0-dd4f-46f1-aaab-61f3a1d554f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:47.975047 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1432/2892] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:47.996601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860cee-3cec-49f2-9808-72d241dd09a6 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:48.008439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97860cee-3cec-49f2-9808-72d241dd09a6 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:48.013876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01b380-6f6f-4cc8-a89a-69e652e84175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:48.014460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01b380-6f6f-4cc8-a89a-69e652e84175 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:48.014984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01b380-6f6f-4cc8-a89a-69e652e84175 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:48.015266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01b380-6f6f-4cc8-a89a-69e652e84175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:48.024292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01b380-6f6f-4cc8-a89a-69e652e84175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:48.025909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01b380-6f6f-4cc8-a89a-69e652e84175 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:48.035116 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1461/2893] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:52:51.051083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fd709-c29b-4553-a028-5f9d9f49ac13 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:51.051574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fd709-c29b-4553-a028-5f9d9f49ac13 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:51.051703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fd709-c29b-4553-a028-5f9d9f49ac13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:51.072451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-264fd709-c29b-4553-a028-5f9d9f49ac13 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:51.073654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-264fd709-c29b-4553-a028-5f9d9f49ac13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:51.081381 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1433/2894] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:51.103204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01b380-6f6f-4cc8-a89a-69e652e84175 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:51.115990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01b380-6f6f-4cc8-a89a-69e652e84175 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:51.122998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7f659-8c94-4a95-ba17-80adf4e0f997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:51.123577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7f659-8c94-4a95-ba17-80adf4e0f997 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:51.123856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7f659-8c94-4a95-ba17-80adf4e0f997 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:51.124128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7f659-8c94-4a95-ba17-80adf4e0f997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:51.133375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7f659-8c94-4a95-ba17-80adf4e0f997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:51.135008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7f659-8c94-4a95-ba17-80adf4e0f997 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:51.144630 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1462/2895] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:55.260667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664146-94e0-4d86-886a-7893d52b895f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:55.261219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664146-94e0-4d86-886a-7893d52b895f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:55.261219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664146-94e0-4d86-886a-7893d52b895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:55.281487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-48664146-94e0-4d86-886a-7893d52b895f None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:55.282769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-48664146-94e0-4d86-886a-7893d52b895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:55.290655 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1434/2896] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:52:55.317515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7f659-8c94-4a95-ba17-80adf4e0f997 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:55.329810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7f659-8c94-4a95-ba17-80adf4e0f997 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:55.336553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecb7b8-646f-4c5d-8e0e-616959e2439e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:55.336769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecb7b8-646f-4c5d-8e0e-616959e2439e service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:55.337007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecb7b8-646f-4c5d-8e0e-616959e2439e service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:55.337276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecb7b8-646f-4c5d-8e0e-616959e2439e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:55.345312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecb7b8-646f-4c5d-8e0e-616959e2439e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:55.347080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecb7b8-646f-4c5d-8e0e-616959e2439e service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:55.356440 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1463/2897] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:52:58.407037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5006f-b705-45d8-9346-7d367645d1dc None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:58.407583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5006f-b705-45d8-9346-7d367645d1dc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:58.407685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5006f-b705-45d8-9346-7d367645d1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:58.428254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-28f5006f-b705-45d8-9346-7d367645d1dc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:52:58.429461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-28f5006f-b705-45d8-9346-7d367645d1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:52:58.437206 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1435/2898] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:52:58.460272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecb7b8-646f-4c5d-8e0e-616959e2439e service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:52:58.471461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecb7b8-646f-4c5d-8e0e-616959e2439e service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:52:58.477454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93457cd-ef16-4364-981c-ca431593e705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:52:58.478083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93457cd-ef16-4364-981c-ca431593e705 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:52:58.478368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93457cd-ef16-4364-981c-ca431593e705 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:52:58.478643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93457cd-ef16-4364-981c-ca431593e705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:52:58.486674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93457cd-ef16-4364-981c-ca431593e705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:52:58.488255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93457cd-ef16-4364-981c-ca431593e705 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:52:58.498173 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1464/2899] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:53:01.510817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5006f-b705-45d8-9346-7d367645d1dc None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:53:01.520640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5006f-b705-45d8-9346-7d367645d1dc None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:53:01.524311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:53:01.524903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:01.525177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:01.525444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:01.527160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:53:01.541329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:53:01.542785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:53:01.554296 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1436/2900] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:53:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:53:08.494136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=70 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:11.548648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=24 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:13.728485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dea0db-e329-48ef-812a-4af9c0dd5333 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:13.729029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dea0db-e329-48ef-812a-4af9c0dd5333 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:13.729029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dea0db-e329-48ef-812a-4af9c0dd5333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:13.748063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e5dea0db-e329-48ef-812a-4af9c0dd5333 None None] MFA Rules not processed for user `7bd88623aa1e43e19db56b4478e1e0ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:53:13.749246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e5dea0db-e329-48ef-812a-4af9c0dd5333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:53:13.756673 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1465/2901] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:53:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:53:13.989683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:53:13.999231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f124a93-92f9-4c2b-b5a8-12e29b1d380e service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:53:14.002777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:53:14.003362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:14.003636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:14.003903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:14.005587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:53:14.015925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:53:14.017357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:53:14.026011 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1437/2902] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:53:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:53:23.743302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:24.022158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:27.664921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bb0c1-e43a-4620-a1f1-d988300f1e76 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:27.665403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bb0c1-e43a-4620-a1f1-d988300f1e76 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:27.665602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bb0c1-e43a-4620-a1f1-d988300f1e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:27.684672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-070bb0c1-e43a-4620-a1f1-d988300f1e76 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:53:27.686014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-070bb0c1-e43a-4620-a1f1-d988300f1e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:53:27.694710 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1466/2903] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:53:27.715862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:53:27.730010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580eaf8-1975-4636-bc1c-4ece4f592d50 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:53:27.735352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17c64-4c95-495a-b5bc-375251d5ec88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:53:27.735939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17c64-4c95-495a-b5bc-375251d5ec88 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:27.736216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17c64-4c95-495a-b5bc-375251d5ec88 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:27.736488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17c64-4c95-495a-b5bc-375251d5ec88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:27.745144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd17c64-4c95-495a-b5bc-375251d5ec88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:53:27.746722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd17c64-4c95-495a-b5bc-375251d5ec88 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:53:27.755838 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1438/2904] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:53:30.919335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189c8f-4d91-43a8-aeda-470816502ecb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:30.919867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189c8f-4d91-43a8-aeda-470816502ecb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:30.919867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189c8f-4d91-43a8-aeda-470816502ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:30.941111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c0189c8f-4d91-43a8-aeda-470816502ecb None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:53:30.942268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0189c8f-4d91-43a8-aeda-470816502ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:53:30.951074 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1467/2905] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:53:30.970247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17c64-4c95-495a-b5bc-375251d5ec88 service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:53:30.982135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17c64-4c95-495a-b5bc-375251d5ec88 service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:53:30.987991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f8264-9486-41f8-a23b-653bbea4db3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:53:30.988596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f8264-9486-41f8-a23b-653bbea4db3f service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:30.988877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f8264-9486-41f8-a23b-653bbea4db3f service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:30.989151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f8264-9486-41f8-a23b-653bbea4db3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:30.998169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f8264-9486-41f8-a23b-653bbea4db3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:53:30.999748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f8264-9486-41f8-a23b-653bbea4db3f service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:53:31.010519 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1439/2906] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:53:34.040694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cff4cf-e557-4467-aa22-5ce2d87fbedc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:34.041215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cff4cf-e557-4467-aa22-5ce2d87fbedc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:34.041215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cff4cf-e557-4467-aa22-5ce2d87fbedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:34.060272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c5cff4cf-e557-4467-aa22-5ce2d87fbedc None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:53:34.061492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c5cff4cf-e557-4467-aa22-5ce2d87fbedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:53:34.069319 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1468/2907] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:53:34.100874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f8264-9486-41f8-a23b-653bbea4db3f service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:53:34.111920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f8264-9486-41f8-a23b-653bbea4db3f service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:53:34.117386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73edc79-f255-4f66-be44-c0fa1fda56cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:53:34.118012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73edc79-f255-4f66-be44-c0fa1fda56cf service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:34.118292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73edc79-f255-4f66-be44-c0fa1fda56cf service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:34.118573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73edc79-f255-4f66-be44-c0fa1fda56cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:34.126606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73edc79-f255-4f66-be44-c0fa1fda56cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:53:34.128169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73edc79-f255-4f66-be44-c0fa1fda56cf service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:53:34.138082 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1440/2908] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:53:37.191437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb9757-9443-4777-9326-c9ebb014f00d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:37.191914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb9757-9443-4777-9326-c9ebb014f00d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:37.192064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb9757-9443-4777-9326-c9ebb014f00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:37.212213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-01bb9757-9443-4777-9326-c9ebb014f00d None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:53:37.213440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-01bb9757-9443-4777-9326-c9ebb014f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:53:37.221377 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1469/2909] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:53:37.243894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73edc79-f255-4f66-be44-c0fa1fda56cf service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:53:37.255304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73edc79-f255-4f66-be44-c0fa1fda56cf service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:53:37.261194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc129ac-914a-423e-b37d-d77d2db575b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:53:37.261823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc129ac-914a-423e-b37d-d77d2db575b6 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:37.262100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc129ac-914a-423e-b37d-d77d2db575b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:37.262368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc129ac-914a-423e-b37d-d77d2db575b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:37.270694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc129ac-914a-423e-b37d-d77d2db575b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:53:37.272617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc129ac-914a-423e-b37d-d77d2db575b6 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:53:37.281854 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1441/2910] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:53:41.395347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327e417-01e7-47bd-a9da-3284da253e71 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:41.395835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327e417-01e7-47bd-a9da-3284da253e71 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:41.395938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327e417-01e7-47bd-a9da-3284da253e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:41.416895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3327e417-01e7-47bd-a9da-3284da253e71 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:53:41.418152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3327e417-01e7-47bd-a9da-3284da253e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:53:41.426067 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1470/2911] 10.222.0.40 () {58 vars in 1016 bytes} [Fri Sep 30 09:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:53:41.447935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc129ac-914a-423e-b37d-d77d2db575b6 service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:53:41.459153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc129ac-914a-423e-b37d-d77d2db575b6 service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:53:41.465363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321021bc-e9f9-4078-968f-2982ecd0040e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:53:41.465987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321021bc-e9f9-4078-968f-2982ecd0040e service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:53:41.466264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321021bc-e9f9-4078-968f-2982ecd0040e service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:53:41.466533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321021bc-e9f9-4078-968f-2982ecd0040e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:53:41.474671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321021bc-e9f9-4078-968f-2982ecd0040e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:53:41.476946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321021bc-e9f9-4078-968f-2982ecd0040e service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:53:41.486098 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1442/2912] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:53:51.411112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=15 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:53:51.482056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=30 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:10.393956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327e417-01e7-47bd-a9da-3284da253e71 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:10.395401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3327e417-01e7-47bd-a9da-3284da253e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:10.460091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321021bc-e9f9-4078-968f-2982ecd0040e service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:10.505348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321021bc-e9f9-4078-968f-2982ecd0040e service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:10.511003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:10.511663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:10.511937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:10.512207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] PATH_INFO: `/v3/users/7bd88623aa1e43e19db56b4478e1e0ad` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:10.513333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd88623aa1e43e19db56b4478e1e0ad)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:10.515568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:10.531172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327e417-01e7-47bd-a9da-3284da253e71 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:10.536377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:10.536965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:10.537246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:10.537525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:10.540301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:10.554622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:10.556986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:10.569508 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1471/2913] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:54:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 30 09:54:10.574648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd88623aa1e43e19db56b4478e1e0ad'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.582387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd88623aa1e43e19db56b4478e1e0ad'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.583077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd88623aa1e43e19db56b4478e1e0ad'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.583686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd88623aa1e43e19db56b4478e1e0ad'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.591517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-28084301-e4ce-44ff-8c83-2d3a4582eee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd88623aa1e43e19db56b4478e1e0ad'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.609747 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1443/2914] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:54:10 2022] DELETE /identity/v3/users/7bd88623aa1e43e19db56b4478e1e0ad => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 30 09:54:10.783556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:10.795280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd1d33-a6c2-4e3e-86c0-cff956903b32 service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:10.854523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:10.855150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:10.855428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:10.855699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] PATH_INFO: `/v3/projects/95afd46a9b5947f9918007a23dffdbce` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:10.856800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95afd46a9b5947f9918007a23dffdbce)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:10.858742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:10.900667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:10.904537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95afd46a9b5947f9918007a23dffdbce', 'request_id': 'req-87ab433e-bedf-4608-b42a-ea41821993d4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.912234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95afd46a9b5947f9918007a23dffdbce', 'request_id': 'req-87ab433e-bedf-4608-b42a-ea41821993d4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.919088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95afd46a9b5947f9918007a23dffdbce', 'request_id': 'req-87ab433e-bedf-4608-b42a-ea41821993d4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.929331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] The token cache is being invalidate because project 95afd46a9b5947f9918007a23dffdbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:54:10.929619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.930264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87ab433e-bedf-4608-b42a-ea41821993d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:10.931734 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1472/2915] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:54:10 2022] DELETE /identity/v3/projects/95afd46a9b5947f9918007a23dffdbce => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:54:10.936744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ab050-8964-4c33-8855-00a6b2ddc000 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:10.937023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ab050-8964-4c33-8855-00a6b2ddc000 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:10.937292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ab050-8964-4c33-8855-00a6b2ddc000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:10.956887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ca8ab050-8964-4c33-8855-00a6b2ddc000 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:10.958142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ca8ab050-8964-4c33-8855-00a6b2ddc000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:11.017918 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1444/2916] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:11.026723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707384-fbdf-47d8-ab0a-f21af07cbcfd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.028958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707384-fbdf-47d8-ab0a-f21af07cbcfd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.029229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed707384-fbdf-47d8-ab0a-f21af07cbcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.050346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ed707384-fbdf-47d8-ab0a-f21af07cbcfd None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:11.051552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ed707384-fbdf-47d8-ab0a-f21af07cbcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:11.059645 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1473/2917] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:11.064425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ab050-8964-4c33-8855-00a6b2ddc000 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:11.077956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ab050-8964-4c33-8855-00a6b2ddc000 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:11.082611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:11.083206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.083482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.083751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.084389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:54:11.084685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:11.086305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:11.098547 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1445/2918] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 30 09:54:11.103601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed707384-fbdf-47d8-ab0a-f21af07cbcfd None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:11.112828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed707384-fbdf-47d8-ab0a-f21af07cbcfd None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:11.117399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c57a08-2207-492f-8fd6-170d25bce7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:11.118110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c57a08-2207-492f-8fd6-170d25bce7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.118391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c57a08-2207-492f-8fd6-170d25bce7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.118662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c57a08-2207-492f-8fd6-170d25bce7d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.119309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c57a08-2207-492f-8fd6-170d25bce7d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:11.121322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c57a08-2207-492f-8fd6-170d25bce7d1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:11.147605 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1474/2919] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:54:11 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:54:11.152189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:11.162114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554b872-d30d-4da6-8cb1-785a18b7ad4a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:11.194441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:11.223024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c27531-10b7-4aae-90c2-55a52c1aede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:11.223628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c27531-10b7-4aae-90c2-55a52c1aede4 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.223904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c27531-10b7-4aae-90c2-55a52c1aede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.224172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c27531-10b7-4aae-90c2-55a52c1aede4 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.224802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c27531-10b7-4aae-90c2-55a52c1aede4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:11.226753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c27531-10b7-4aae-90c2-55a52c1aede4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:11.248051 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1446/2920] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:54:11 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:54:11.252151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c57a08-2207-492f-8fd6-170d25bce7d1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:11.262851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c57a08-2207-492f-8fd6-170d25bce7d1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:11.267667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b8bc6-f7ec-470f-908b-68599a83be88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:11.268260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b8bc6-f7ec-470f-908b-68599a83be88 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.268541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b8bc6-f7ec-470f-908b-68599a83be88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.268813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b8bc6-f7ec-470f-908b-68599a83be88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.269450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b8bc6-f7ec-470f-908b-68599a83be88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:11.271117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b8bc6-f7ec-470f-908b-68599a83be88 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:11.282286 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1475/2921] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:54:11.286151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c27531-10b7-4aae-90c2-55a52c1aede4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:11.296040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c27531-10b7-4aae-90c2-55a52c1aede4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:11.300589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99450548-92d6-4d2d-ad7a-15554adda3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:11.301177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450548-92d6-4d2d-ad7a-15554adda3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.301460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450548-92d6-4d2d-ad7a-15554adda3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.301765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450548-92d6-4d2d-ad7a-15554adda3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.302404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99450548-92d6-4d2d-ad7a-15554adda3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:11.303981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99450548-92d6-4d2d-ad7a-15554adda3a1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:11.313676 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1447/2922] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:54:11.317652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b8bc6-f7ec-470f-908b-68599a83be88 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:11.328322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b8bc6-f7ec-470f-908b-68599a83be88 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:11.332840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:11.333476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.333805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.334084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] PATH_INFO: `/v3/projects/6d78ee5c711841ab80db5935a22cb4c5/users/2f4dfa969a9b49cb9ec8c22780e67610/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.351731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d78ee5c711841ab80db5935a22cb4c5, user_id=2f4dfa969a9b49cb9ec8c22780e67610, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:11.356070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:11.368949 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1476/2923] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:54:11 2022] PUT /identity/v3/projects/6d78ee5c711841ab80db5935a22cb4c5/users/2f4dfa969a9b49cb9ec8c22780e67610/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:54:11.379286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49536a0b-2541-4dc1-9e45-7a15e8259b64 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.380112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49536a0b-2541-4dc1-9e45-7a15e8259b64 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.380387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49536a0b-2541-4dc1-9e45-7a15e8259b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.411054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-49536a0b-2541-4dc1-9e45-7a15e8259b64 None None] MFA Rules not processed for user `2f4dfa969a9b49cb9ec8c22780e67610`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:11.412278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-49536a0b-2541-4dc1-9e45-7a15e8259b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:11.476747 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1448/2924] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:11.487569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:11.489351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:11.546100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd60be-f98d-4bdc-a826-a465170fb59e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:11.549799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b9dbd-8598-447e-9bd7-6e45e263266f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:11.550421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b9dbd-8598-447e-9bd7-6e45e263266f service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:11.550699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b9dbd-8598-447e-9bd7-6e45e263266f service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:11.550998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b9dbd-8598-447e-9bd7-6e45e263266f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:11.560001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b9dbd-8598-447e-9bd7-6e45e263266f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:11.561436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b9dbd-8598-447e-9bd7-6e45e263266f service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:11.570295 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1477/2925] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:54:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:54:21.473973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:21.566183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:21.630371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49536a0b-2541-4dc1-9e45-7a15e8259b64 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:21.710682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49536a0b-2541-4dc1-9e45-7a15e8259b64 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:21.715403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:21.716027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:21.716305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:21.716577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] PATH_INFO: `/v3/users/2f4dfa969a9b49cb9ec8c22780e67610` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:21.717709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4dfa969a9b49cb9ec8c22780e67610)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:21.719614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:21.765481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4dfa969a9b49cb9ec8c22780e67610'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:21.772287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4dfa969a9b49cb9ec8c22780e67610'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:21.772947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4dfa969a9b49cb9ec8c22780e67610'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:21.773560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4dfa969a9b49cb9ec8c22780e67610'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:21.781048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4dfa969a9b49cb9ec8c22780e67610'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:21.798288 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1449/2926] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:54:21 2022] DELETE /identity/v3/users/2f4dfa969a9b49cb9ec8c22780e67610 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:54:21.802886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec441b7-efaf-4dd8-b6eb-492a606c1cd2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:21.803171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec441b7-efaf-4dd8-b6eb-492a606c1cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:21.803439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec441b7-efaf-4dd8-b6eb-492a606c1cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:21.842610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-aec441b7-efaf-4dd8-b6eb-492a606c1cd2 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:21.843816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aec441b7-efaf-4dd8-b6eb-492a606c1cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:21.900616 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1478/2927] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:21.910630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:21.912047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:21.980335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:21.990928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc6dc2-ad89-467b-8c66-e43ceb6f652c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:21.995159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52b226-d213-4fb4-8618-e6c6cbebb7aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:21.995870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b226-d213-4fb4-8618-e6c6cbebb7aa service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:21.996153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b226-d213-4fb4-8618-e6c6cbebb7aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:21.996432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52b226-d213-4fb4-8618-e6c6cbebb7aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.005335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52b226-d213-4fb4-8618-e6c6cbebb7aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.006813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52b226-d213-4fb4-8618-e6c6cbebb7aa service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.017006 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1450/2928] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:54:22.283219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec441b7-efaf-4dd8-b6eb-492a606c1cd2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:22.284699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aec441b7-efaf-4dd8-b6eb-492a606c1cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:22.299910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec441b7-efaf-4dd8-b6eb-492a606c1cd2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:22.305217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:22.306248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.306490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.306769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] PATH_INFO: `/v3/projects/6d78ee5c711841ab80db5935a22cb4c5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.308683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d78ee5c711841ab80db5935a22cb4c5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.310677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.358043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d78ee5c711841ab80db5935a22cb4c5', 'request_id': 'req-8365026b-11ed-4aae-8dd1-8964e8b24499'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:22.365261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d78ee5c711841ab80db5935a22cb4c5', 'request_id': 'req-8365026b-11ed-4aae-8dd1-8964e8b24499'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:22.371472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d78ee5c711841ab80db5935a22cb4c5', 'request_id': 'req-8365026b-11ed-4aae-8dd1-8964e8b24499'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:22.381601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] The token cache is being invalidate because project 6d78ee5c711841ab80db5935a22cb4c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:54:22.381918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:22.382542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8365026b-11ed-4aae-8dd1-8964e8b24499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:54:22.384009 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1479/2929] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:54:22 2022] DELETE /identity/v3/projects/6d78ee5c711841ab80db5935a22cb4c5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:54:22.390581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed4dc-417b-4ab5-b2f5-c129052a173a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.390873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed4dc-417b-4ab5-b2f5-c129052a173a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.391148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ed4dc-417b-4ab5-b2f5-c129052a173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.421459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7e1ed4dc-417b-4ab5-b2f5-c129052a173a None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:22.422687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7e1ed4dc-417b-4ab5-b2f5-c129052a173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:22.482325 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1451/2930] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:22.489987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff90892-9486-4c95-be1d-4820e0fba18b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.490278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff90892-9486-4c95-be1d-4820e0fba18b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.490558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff90892-9486-4c95-be1d-4820e0fba18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.510957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8ff90892-9486-4c95-be1d-4820e0fba18b None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:22.512187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8ff90892-9486-4c95-be1d-4820e0fba18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:22.520335 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1480/2931] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:22.524413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ed4dc-417b-4ab5-b2f5-c129052a173a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:22.534256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ed4dc-417b-4ab5-b2f5-c129052a173a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:22.538799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:22.539393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.539675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.539952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.540595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 30 09:54:22.540898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.542821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.554252 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1452/2932] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 30 09:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 30 09:54:22.558838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff90892-9486-4c95-be1d-4820e0fba18b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:22.568120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff90892-9486-4c95-be1d-4820e0fba18b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:22.572807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bc3d9-a587-451e-8d7c-43ab96cfb803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:22.574096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bc3d9-a587-451e-8d7c-43ab96cfb803 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.574374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bc3d9-a587-451e-8d7c-43ab96cfb803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.574645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bc3d9-a587-451e-8d7c-43ab96cfb803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.575274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bc3d9-a587-451e-8d7c-43ab96cfb803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.577480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bc3d9-a587-451e-8d7c-43ab96cfb803 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.605605 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1481/2933] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:54:22 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:54:22.609801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:22.619522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc1f9d-aef1-46ea-938c-ed55ade9e7f2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:22.674786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca420-20c9-49be-81b9-45230129beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:22.675383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca420-20c9-49be-81b9-45230129beec admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.675660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca420-20c9-49be-81b9-45230129beec admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.675930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca420-20c9-49be-81b9-45230129beec admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.676557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca420-20c9-49be-81b9-45230129beec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.678489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca420-20c9-49be-81b9-45230129beec admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.699875 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1453/2934] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:54:22 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:54:22.703943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bc3d9-a587-451e-8d7c-43ab96cfb803 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:22.714222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bc3d9-a587-451e-8d7c-43ab96cfb803 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:22.718981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81065e9b-decf-43b3-907a-2a27f6aad80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:22.719576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81065e9b-decf-43b3-907a-2a27f6aad80f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.719856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81065e9b-decf-43b3-907a-2a27f6aad80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.720122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81065e9b-decf-43b3-907a-2a27f6aad80f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.720759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81065e9b-decf-43b3-907a-2a27f6aad80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.722423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81065e9b-decf-43b3-907a-2a27f6aad80f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.732320 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1482/2935] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:54:22.736086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca420-20c9-49be-81b9-45230129beec admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:22.746424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca420-20c9-49be-81b9-45230129beec admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:22.750993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcbb29-4c28-4068-bf79-5d26e09016ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:22.751573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcbb29-4c28-4068-bf79-5d26e09016ba admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.751849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcbb29-4c28-4068-bf79-5d26e09016ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.752116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcbb29-4c28-4068-bf79-5d26e09016ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.752748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcbb29-4c28-4068-bf79-5d26e09016ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.754369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcbb29-4c28-4068-bf79-5d26e09016ba admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.765637 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1454/2936] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:54:22.769614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81065e9b-decf-43b3-907a-2a27f6aad80f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:22.779532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81065e9b-decf-43b3-907a-2a27f6aad80f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:22.784033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:22.784669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.784945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.785213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] PATH_INFO: `/v3/projects/85893bbff42c409483a5123cddc4a776/users/4a49491c64d64048ba5cdeccee625720/roles/656b8aa0fc5e4f97b03a4251472f521b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.799753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85893bbff42c409483a5123cddc4a776, user_id=4a49491c64d64048ba5cdeccee625720, role_id=656b8aa0fc5e4f97b03a4251472f521b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.802468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.816160 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1483/2937] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:54:22 2022] PUT /identity/v3/projects/85893bbff42c409483a5123cddc4a776/users/4a49491c64d64048ba5cdeccee625720/roles/656b8aa0fc5e4f97b03a4251472f521b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:54:22.821183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8409e8-7a50-4a1a-a63e-9edcef6e1d06 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.821465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8409e8-7a50-4a1a-a63e-9edcef6e1d06 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.821791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8409e8-7a50-4a1a-a63e-9edcef6e1d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.840630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7e8409e8-7a50-4a1a-a63e-9edcef6e1d06 None None] MFA Rules not processed for user `22586ccbb1964437ab05abad3612640c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:22.841844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7e8409e8-7a50-4a1a-a63e-9edcef6e1d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:22.879817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:22.902763 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1455/2938] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:54:22.912408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:22.914081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:22.919503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:22.973880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84dea2a-5d6f-4bc6-b5ec-c4895b101cc4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:22.977438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277895c1-2d50-4f00-88a1-375dd45ae058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:22.978076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277895c1-2d50-4f00-88a1-375dd45ae058 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:22.978357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277895c1-2d50-4f00-88a1-375dd45ae058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:22.978629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277895c1-2d50-4f00-88a1-375dd45ae058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:22.988154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277895c1-2d50-4f00-88a1-375dd45ae058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:22.989587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277895c1-2d50-4f00-88a1-375dd45ae058 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:22.999710 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1484/2939] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:54:26.977902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7649d3-1d79-43ce-bf61-510e82953c4a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:26.978386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7649d3-1d79-43ce-bf61-510e82953c4a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:26.978515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7649d3-1d79-43ce-bf61-510e82953c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:27.008768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ad7649d3-1d79-43ce-bf61-510e82953c4a None None] MFA Rules not processed for user `4a49491c64d64048ba5cdeccee625720`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:27.010032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ad7649d3-1d79-43ce-bf61-510e82953c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:27.085478 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1456/2940] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:27.090159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277895c1-2d50-4f00-88a1-375dd45ae058 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:27.099979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277895c1-2d50-4f00-88a1-375dd45ae058 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:27.105868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b338a5-b677-48c8-84cf-e2a22c8a91a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:27.106478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b338a5-b677-48c8-84cf-e2a22c8a91a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:27.106907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b338a5-b677-48c8-84cf-e2a22c8a91a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:27.107190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b338a5-b677-48c8-84cf-e2a22c8a91a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:27.107830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b338a5-b677-48c8-84cf-e2a22c8a91a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:27.109819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b338a5-b677-48c8-84cf-e2a22c8a91a8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:27.138704 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1485/2941] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 30 09:54:27 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:54:27.142967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7649d3-1d79-43ce-bf61-510e82953c4a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:27.154503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7649d3-1d79-43ce-bf61-510e82953c4a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:27.216875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1683d68-cdb8-4553-a511-8ab165b9b364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:27.217495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1683d68-cdb8-4553-a511-8ab165b9b364 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:27.217802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1683d68-cdb8-4553-a511-8ab165b9b364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:27.218072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1683d68-cdb8-4553-a511-8ab165b9b364 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:27.218864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1683d68-cdb8-4553-a511-8ab165b9b364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:27.220790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1683d68-cdb8-4553-a511-8ab165b9b364 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:27.248270 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1457/2942] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 30 09:54:27 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 30 09:54:27.249979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b338a5-b677-48c8-84cf-e2a22c8a91a8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:27.262378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b338a5-b677-48c8-84cf-e2a22c8a91a8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:27.267230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dc483-7cf9-40ff-b2c8-334960ad19b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:27.267836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dc483-7cf9-40ff-b2c8-334960ad19b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:27.268116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dc483-7cf9-40ff-b2c8-334960ad19b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:27.268387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dc483-7cf9-40ff-b2c8-334960ad19b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:27.269027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dc483-7cf9-40ff-b2c8-334960ad19b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:27.270673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dc483-7cf9-40ff-b2c8-334960ad19b1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:27.283272 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1486/2943] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:54:27.287374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1683d68-cdb8-4553-a511-8ab165b9b364 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:27.297773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1683d68-cdb8-4553-a511-8ab165b9b364 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:27.302455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b382e00-9aa2-429e-a11f-2af681352586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:27.303123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382e00-9aa2-429e-a11f-2af681352586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:27.303404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382e00-9aa2-429e-a11f-2af681352586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:27.303686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382e00-9aa2-429e-a11f-2af681352586 admin admin] PATH_INFO: `/v3/projects/ba37653fe3b843e7904b39bf6d61f667/users/5ba427e7442e49deae0ae081a7028990/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:27.319978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b382e00-9aa2-429e-a11f-2af681352586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba37653fe3b843e7904b39bf6d61f667, user_id=5ba427e7442e49deae0ae081a7028990, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:27.322774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b382e00-9aa2-429e-a11f-2af681352586 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:27.338073 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1458/2944] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:54:27 2022] PUT /identity/v3/projects/ba37653fe3b843e7904b39bf6d61f667/users/5ba427e7442e49deae0ae081a7028990/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:54:27.342566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dc483-7cf9-40ff-b2c8-334960ad19b1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:27.352879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dc483-7cf9-40ff-b2c8-334960ad19b1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:27.428800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101878b6-ba69-4b46-a607-dedbb3701f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:27.429429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101878b6-ba69-4b46-a607-dedbb3701f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:27.429741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101878b6-ba69-4b46-a607-dedbb3701f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:27.430018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101878b6-ba69-4b46-a607-dedbb3701f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:27.430667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101878b6-ba69-4b46-a607-dedbb3701f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:27.432823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101878b6-ba69-4b46-a607-dedbb3701f17 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:27.447258 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1487/2945] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 30 09:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 30 09:54:27.452175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b382e00-9aa2-429e-a11f-2af681352586 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:27.464105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b382e00-9aa2-429e-a11f-2af681352586 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:27.468846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:27.469533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:27.469871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:27.470146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] PATH_INFO: `/v3/projects/ba37653fe3b843e7904b39bf6d61f667/users/5ba427e7442e49deae0ae081a7028990/roles/7beac02fae504326b332fd7fbdf80d35` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:27.472094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba37653fe3b843e7904b39bf6d61f667, user_id=5ba427e7442e49deae0ae081a7028990, role_id=7beac02fae504326b332fd7fbdf80d35)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:27.474902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:27.487965 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1459/2946] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 30 09:54:27 2022] PUT /identity/v3/projects/ba37653fe3b843e7904b39bf6d61f667/users/5ba427e7442e49deae0ae081a7028990/roles/7beac02fae504326b332fd7fbdf80d35 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:54:32.028258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1467c-a38f-4ed9-9282-1eee2ae2739b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:32.028701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1467c-a38f-4ed9-9282-1eee2ae2739b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:32.028942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1467c-a38f-4ed9-9282-1eee2ae2739b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:32.062735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3ad1467c-a38f-4ed9-9282-1eee2ae2739b None None] MFA Rules not processed for user `5ba427e7442e49deae0ae081a7028990`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:32.063970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ad1467c-a38f-4ed9-9282-1eee2ae2739b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:32.139620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:32.157875 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1488/2947] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 30 09:54:32.172016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:32.174100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:32.213330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:32.284923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b252d-d3ab-4dd7-aee3-0f93ebf25e42 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:32.289047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fe7a090bd7941bb827d301aa4949857', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:32.289696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:32.289979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:32.290257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:32.355766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:32.357365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:32.369812 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1460/2948] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:54:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:54:32.504610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafb207-73be-44dc-9484-fabf857e288a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:32.504899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafb207-73be-44dc-9484-fabf857e288a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:32.505201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafb207-73be-44dc-9484-fabf857e288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:32.564661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7cafb207-73be-44dc-9484-fabf857e288a None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:32.565927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7cafb207-73be-44dc-9484-fabf857e288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:32.637340 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1489/2949] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:54:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:32.688210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:32.689644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:32.780867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac2117-5c43-48bb-9095-80f6987bdd73 service glance] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:32.785985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d86f1c-51dc-4300-93ef-5e5fc3876fd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:32.786591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d86f1c-51dc-4300-93ef-5e5fc3876fd9 service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:32.786877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d86f1c-51dc-4300-93ef-5e5fc3876fd9 service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:32.787178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d86f1c-51dc-4300-93ef-5e5fc3876fd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:32.796140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d86f1c-51dc-4300-93ef-5e5fc3876fd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:32.797745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d86f1c-51dc-4300-93ef-5e5fc3876fd9 service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:32.805527 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1461/2950] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:54:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:54:42.544381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530fa47-5083-4180-a9b1-1c22d51fdd5e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:42.544890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530fa47-5083-4180-a9b1-1c22d51fdd5e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:42.544992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530fa47-5083-4180-a9b1-1c22d51fdd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:42.563479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6530fa47-5083-4180-a9b1-1c22d51fdd5e None None] MFA Rules not processed for user `4a49491c64d64048ba5cdeccee625720`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:42.564661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6530fa47-5083-4180-a9b1-1c22d51fdd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:42.571386 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1490/2951] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 30 09:54:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:42.802436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=87 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:44.476569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a98be7-d1f6-43d8-afdf-af8ff23a5b1b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:44.477180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a98be7-d1f6-43d8-afdf-af8ff23a5b1b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:44.477180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a98be7-d1f6-43d8-afdf-af8ff23a5b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:44.497252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-99a98be7-d1f6-43d8-afdf-af8ff23a5b1b None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:54:44.498583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-99a98be7-d1f6-43d8-afdf-af8ff23a5b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:54:44.506103 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1462/2952] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 30 09:54:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:54:44.519693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530fa47-5083-4180-a9b1-1c22d51fdd5e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:54:44.532415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530fa47-5083-4180-a9b1-1c22d51fdd5e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:54:44.537615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:54:44.538256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:54:44.538532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:54:44.538804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:54:44.547460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:54:44.549045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:54:44.558546 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1491/2953] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:54:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:54:54.491824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:54:54.554422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=59 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:08.483900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a98be7-d1f6-43d8-afdf-af8ff23a5b1b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:55:08.485287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-99a98be7-d1f6-43d8-afdf-af8ff23a5b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:55:08.540527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a98be7-d1f6-43d8-afdf-af8ff23a5b1b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:55:08.544493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:55:08.545098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:55:08.545379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:55:08.545680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:55:08.554448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:55:08.555861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:55:08.566035 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1463/2954] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:55:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:55:10.337448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:55:10.339109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:55:10.459895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd9f29-ee02-46a4-a39e-63358afb9baf service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:55:10.469643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:55:10.470446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:55:10.470734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:55:10.471007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:55:10.472896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:55:10.490238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:55:10.491893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:55:10.506506 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1492/2955] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:55:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:55:13.575357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:55:13.576752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:55:13.665522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba357070-cd29-4512-8b82-5396408223ce service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:55:13.671027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:55:13.671629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:55:13.671909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:55:13.672182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:55:13.673922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:55:13.685780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:55:13.687814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:55:13.699119 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1464/2956] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:55:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:55:20.500746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=43 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:55:23.693677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=67 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:20.260740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:20.270835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8e9ca-c51a-407c-8447-9199c47f4461 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:20.274326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f41e236339784d169258f06f1090b7f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:20.274907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:20.275184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:20.275450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:20.277741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:20.287128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:20.288546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:20.298321 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1493/2957] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:57:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:57:30.294555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:37.915104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:37.916858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:37.936835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17388d75-1744-4f90-b03d-b8323ebcc8c3 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:37.945856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88472db6-8ad8-406b-b5a3-1dac3ae322b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa055ee122ce4e92b4da7f8222521a9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:37.947196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88472db6-8ad8-406b-b5a3-1dac3ae322b9 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:37.948430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88472db6-8ad8-406b-b5a3-1dac3ae322b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:37.948927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88472db6-8ad8-406b-b5a3-1dac3ae322b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:37.951683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-88472db6-8ad8-406b-b5a3-1dac3ae322b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:37.969180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88472db6-8ad8-406b-b5a3-1dac3ae322b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:37.971766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88472db6-8ad8-406b-b5a3-1dac3ae322b9 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:37.989233 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1465/2958] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:57:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:57:39.033959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:39.035528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:39.117642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68289d-bec2-474f-be32-839fd2588f74 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:39.122656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c1784d615d8439a9a4cb9fea305a41b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:39.123295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:39.123571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:39.123843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:39.125656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:39.140058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:39.141493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:39.153528 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1494/2959] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:57:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:57:45.374585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55744-704a-47ce-8873-d7f22a40ada7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:45.375046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55744-704a-47ce-8873-d7f22a40ada7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:45.375167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c55744-704a-47ce-8873-d7f22a40ada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:45.395521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-96c55744-704a-47ce-8873-d7f22a40ada7 None None] MFA Rules not processed for user `29179e1dc1b045a08cbd7feadc2ececd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 30 09:57:45.396716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-96c55744-704a-47ce-8873-d7f22a40ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:45.404291 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1466/2960] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 30 09:57:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 30 09:57:45.444610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:45.456363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d70013-02b9-462a-a49d-03e37a22671a service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:45.461546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e39c6-1357-4fec-9f14-57be013d9c5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60761ed7c27f4018a6b4791b3b060bb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:45.462185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e39c6-1357-4fec-9f14-57be013d9c5f service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:45.462468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e39c6-1357-4fec-9f14-57be013d9c5f service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:45.462733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e39c6-1357-4fec-9f14-57be013d9c5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:45.471095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e39c6-1357-4fec-9f14-57be013d9c5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:45.472669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e39c6-1357-4fec-9f14-57be013d9c5f service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:45.481198 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1495/2961] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 30 09:57:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 30 09:57:48.919515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c55744-704a-47ce-8873-d7f22a40ada7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:48.932245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c55744-704a-47ce-8873-d7f22a40ada7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:48.937273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:48.937982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:48.938325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:48.938593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:48.940660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:48.954878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:48.957123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:48.977763 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1467/2962] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 09:57:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 09:57:52.079675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e39c6-1357-4fec-9f14-57be013d9c5f service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:52.142860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e39c6-1357-4fec-9f14-57be013d9c5f service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:52.147444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:52.148076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:52.148360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:52.148633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] PATH_INFO: `/v3/users/4a49491c64d64048ba5cdeccee625720` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:52.149826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a49491c64d64048ba5cdeccee625720)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:52.151921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:52.199823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a49491c64d64048ba5cdeccee625720'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.206603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a49491c64d64048ba5cdeccee625720'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.207273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a49491c64d64048ba5cdeccee625720'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.207884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a49491c64d64048ba5cdeccee625720'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.215163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a49491c64d64048ba5cdeccee625720'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.222167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:52.235687 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1496/2963] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:57:52 2022] DELETE /identity/v3/users/4a49491c64d64048ba5cdeccee625720 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:57:52.402895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:52.414829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812dbd8-7bcb-4e6a-8209-13bf0c49f790 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:52.472250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:52.472808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:52.473083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:52.473354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] PATH_INFO: `/v3/projects/85893bbff42c409483a5123cddc4a776` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:52.474498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=85893bbff42c409483a5123cddc4a776)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:52.476396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:52.523509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85893bbff42c409483a5123cddc4a776', 'request_id': 'req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.532155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85893bbff42c409483a5123cddc4a776', 'request_id': 'req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.538312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85893bbff42c409483a5123cddc4a776', 'request_id': 'req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.549146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] The token cache is being invalidate because project 85893bbff42c409483a5123cddc4a776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:57:52.549437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.550103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.551700 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1468/2964] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:57:52 2022] DELETE /identity/v3/projects/85893bbff42c409483a5123cddc4a776 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:57:52.555886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:52.558358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:52.571287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f104da3-2eeb-4818-9f69-17bf7603174e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:52.627256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:52.627879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:52.628180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:52.628448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] PATH_INFO: `/v3/users/5ba427e7442e49deae0ae081a7028990` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:52.629586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba427e7442e49deae0ae081a7028990)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:52.631511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:52.676355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba427e7442e49deae0ae081a7028990'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.683143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba427e7442e49deae0ae081a7028990'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.683797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba427e7442e49deae0ae081a7028990'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.684397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba427e7442e49deae0ae081a7028990'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.691653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba427e7442e49deae0ae081a7028990'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:52.709147 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1497/2965] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 30 09:57:52 2022] DELETE /identity/v3/users/5ba427e7442e49deae0ae081a7028990 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:57:52.881942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 09:57:52.883401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 09:57:52.897051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f36885-f30b-46f4-bb0c-5c4da1a34f8e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 09:57:52.949912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:57:52.956427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22586ccbb1964437ab05abad3612640c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9494127904354d908def6993c27761dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 09:57:52.957319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 09:57:52.957850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 09:57:52.958589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] PATH_INFO: `/v3/projects/ba37653fe3b843e7904b39bf6d61f667` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 09:57:52.959916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba37653fe3b843e7904b39bf6d61f667)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 09:57:52.962457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 09:57:53.009282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba37653fe3b843e7904b39bf6d61f667', 'request_id': 'req-1a3c8422-2684-461f-87bf-a3eb67c91cc8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:53.017162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba37653fe3b843e7904b39bf6d61f667', 'request_id': 'req-1a3c8422-2684-461f-87bf-a3eb67c91cc8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:53.023398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba37653fe3b843e7904b39bf6d61f667', 'request_id': 'req-1a3c8422-2684-461f-87bf-a3eb67c91cc8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:53.033852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] The token cache is being invalidate because project ba37653fe3b843e7904b39bf6d61f667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 30 09:57:53.034142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:53.034794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1a3c8422-2684-461f-87bf-a3eb67c91cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 30 09:57:53.036259 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1469/2966] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 30 09:57:52 2022] DELETE /identity/v3/projects/ba37653fe3b843e7904b39bf6d61f667 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 30 09:58:02.705805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=57,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 09:58:03.030784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 30 10:00:19.550224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 30 10:00:19.551633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 10:00:19.620800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e902b4-3c81-4e73-817c-dc0fe7aa6f86 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 30 10:00:19.680341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde4be37-9985-49aa-a60d-02f7633886f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5989c8a1e28a43d3bc16970850ce49f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3b2bf7840c042eeaf27440952d4aa8f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 30 10:00:19.680869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4be37-9985-49aa-a60d-02f7633886f1 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 30 10:00:19.681151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4be37-9985-49aa-a60d-02f7633886f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 30 10:00:19.681425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4be37-9985-49aa-a60d-02f7633886f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 30 10:00:19.683212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dde4be37-9985-49aa-a60d-02f7633886f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 30 10:00:19.694636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4be37-9985-49aa-a60d-02f7633886f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 30 10:00:19.696225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde4be37-9985-49aa-a60d-02f7633886f1 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 30 10:00:19.706048 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1498/2967] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 30 10:00:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 30 10:00:29.701831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=58 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}